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PREFACE 


The material in this book was originally prepared for an introductory 
course in linear programming given at the Graduate School, U.S. Depart- 
ment of Agriculture, Washington, D.C. In developing and expanding 
the notes into a suitable text, I have attempted to pursue the same objec- 
tives that guided the presentation of the course material. These basic 
aims were to instill in the student an ability to recognize potential 
linear-programming problems, to formulate such problems as linear- 
programming models, to employ the proper computational techniques to 
solve these problems, and to understand the mathematical aspects that 
tie together these elements of linear programming. 

Itis very convenient to divide the subject matter of linear program- 
ming into three separate, but not distinct, areas: theoretical, computational, 
and applied. In teaching the course, I found it appropriate, instructive, 
and beneficial to the students to interlace material from all three areas 
as much as possible. Hence, after an introductory lecture on applica- 
tions and the mathematical model of linear programming (Chap. 1), 
the mathematies of convex sets and linear inequalities were developed 
and followed by the computational aspects of the elimination method for 
solving linear equations (Chap. 2). The mathematical properties of à 
solution to the general linear-programming problem were next evolved. 
Then, in an attempt to explain fully the fundamentals of the simplex 
computational procedure, the ability to generate extreme-point solutions 
was shown to be a simple variation of the elimination technique of Jordan 
and Gauss (Chap. 3). The next set of lectures developed the theoretical 
and computational elements of the simplex method of G. B. Dantzig 
(Chap. 4). A discussion on the duality problems of linear programming 
(Chap. 5) was followed by lectures on the formulation of certain illustra- 
tive applications (Chaps. 10 and 11). "The final lectures of the course! 


1 This one-semester course consisted of 16 evening lectures of 214 hours' duration. 
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described the relationship between linear programming and the zero-sum 
two-person game (Chap. 12). 

'T he basic-course notes have been revised to include full discussion of 
the revised simplex method (Chap. 6), degeneracy procedures (Chap. 7), 
parametric programming (Chap. 8), further computational techniques 
(Chap. 9), and other topies and applications. As a result of a suggestion 
to the author, all the material has been gathered into three parts: an 
introduction, methods, both theoretical and computational, and applica- 
tions.! It is felt that this arrangement enhances the usefulness of the 
book for reference, and presents the three areas of linear programming 
in a related but separate manner. Consequently, the reader will find 
some relatively advanced topies, such as the revised simplex method and 
parametric linear programming, appearing before the basie discussions 
of the transportation problem and general applications. It is suggested 
that, in order to motivate one's study of linear programming, the chapters 
should not be studied in numerical sequence. Instead, one should, as 
soon as possible (probably after Chap. 4 or 5), become acquainted with 
material in the applications sections.? 

Itis felt that the material covered in this text is appropriate for use in 
mathematies courses at the senior or first-year-graduate level. However, 
beeause of the interest in linear-programming methods outside the 
academie field, it seemed advisable to include sufficient material on 
matrices and vectors to make the work complete for all readers (Chap. 2). 
It should be noted that much of the mathematical notation used in 
subsequent chapters is developed in Chap. 2.? 

'The search for the best, the maximum, the minimum, or, in general, 
the optimum solutions to à variety of problems has entertained and 
intrigued man throughout the ages. Euclid, in Book III, was concerned 
with finding the greatest and least straight lines that can be drawn from 
à point to the cireumference of a circle, and in Book IV he described how 
to find the parallelogram of maximum area with a given perimeter. How- 
ever, the rigorous approach to these and more sophisticated problems had 
to wait until the great mathematicians of the seventeenth and eighteenth 
centuries developed the powerful methods of the ealeulus and the caleulus 
of variations. With these techniques we ean find the maximum and 
minimum solutions to a wide range of optimization problems. These 


! This edition includes an additional part on nonlinear programming, Chap. 13. 

? In à course that meets three times a week, one leeture could be devoted to 
applications and /or reports of case studies cited in the Bibliography. 

* The student will soon find that one of the main difficulties in understanding the 
mathematics of linear programming arises from the diverse and often intricate nota- 
tion used in many of the source papers of this field. Wherever possible, I have 
employed "standard," consistent, and, I hope, explicit notation. 
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and other mathematical optimization procedures were mainly concerned 
with the solutions to problems of a geometrie, dynamie, or physical 
nature. Such problems as finding the minimum curves of revolution 
and the curve of quickest descent are resolved by these classical opti- 
mization methods. 

Recently, a new class of optimization problems has originated out of 
the complex organizational structures that permeate modern society. 
Here we are concerned with such matters as the most efficient manner in 
which to run an economy, or the optimum deployment of aircraft that 
maximizes a country's chances of winning a war, or with such mundane 
tasks as mixing the ingredients of a fertilizer to meet agricultural specifica- 
tions at à minimum cost. Research on how to formulate and solve such 
problems has led to the development of new and important optimization 
techniques. Among these we find the subject of this book—Unear 
programming. "The linear-programming model, i.e, the optimization 
of a linear function subject to linear constraints, is simple in its mathe- 
matical structure but powerful in its adaptability to a wide range of 
applications. 

Historically, the general problem of linear programming was first 
developed and applied in 1947 by George B. Dantzig, Marshall Wood, 
and their associates of the U.S. Department of the Air Force. At that 
time, this group was called on to investigate the feasibility of applying 
mathematical and related techniques to military programming and 
planning problems. This inquiry led Dantzig to propose ''that interrela- 
tions between activities of a large organization be viewed as a linear 
programming type model and the optimizing program determined by 
minimizing a linear objective function." In order to develop and extend 
these ideas further, the Air Force organized a research group under the 
title of Project SCOOP (Scientifie Computation of Optimum Programs). 
Besides putting the Air Force programming and budgeting problems on 
a more scientific basis, Project SCOOP's major contribution was the 
formal development and application of the linear-programming model. 
These early applications of linear-programming methods fell into three 
major categories: military applications generated by Project SCOOP, 
interindustry economies based on the Leontief input-output model, and 
problems involving the relationship between zero-sum two-person games 
and linear programming. In the past years these areas of applications 
have been extended and developed, but the main emphasis in linear- 
programming applications has shifted to the general industrial area. 

The initial mathematical statement of the general problem of linear 
programming was made by Dantzig in 1947 along with the simplex 
method, a systematie procedure for solving the problem. Prior to this 
a number of problems (some unsolved) were recognized as being of the 


: PREFACE 


type that dealt with the optimization of a linear function subject to 
linear constraints. "The more important examples include the transporta- 
tion problem posed by Hitchcock (1941) and independently by Ikoopmans 
(1947) and the diet problem of Stigler (1945). The first successful solu- 
tion of a linear-programming problem on a high-speed electronic computer 
occurred in January, 1952, on the National Bureau of Standards SEAC 
machine. Since that time, the simplex algorithm, or variations of this 
procedure, has been coded for most intermediate and large general-purpose 
electronic computers. 

Linear programming has become an important tool of modern theo- 
retical and applied mathematies. "This remarkable growth can be traced 
to the pioneering efforts of many individuals and research organizations. 
Specifically, I would like to make special mention of George B. Dantzig, 
Murray A. Geisler, Leon Goldstein, Julian L. Holley, Walter W. Jacobs, 
Alex Orden, Emil D. Schell, and Marshall K. Wood, all formerly with the 
U.S. Department of the Air Force; Leon Gainen, Alan J. Hoffman, and 
Solomon Pollack, formerly with the National Bureau of Standards; and 
the research groups of the Graduate School of Industria] Administration 
of the Carnegie Institute of Technology, The RAND Corporation, the 
Department of Mathematics of Princeton University, and the Cowles 
Commission for Research in Economies.  Í would like to thank the above 
named individuals and groups, other authors, and their publishers for 
their kind permission to use certain basic material contained in what 
might be considered source documents" of linear programming. | Appro- 
priate references are given in the text. 

This edition includes new sections on network problems and the 
bounded-variable problem, à new chapter on nonlinear programming 
(Part IV), and a revised section on the decomposition algorithm. — Addi- 
tional material and exercises have been included in à number of sections 
and many new publications have been included in the bibliography and 
references. 

I wish to acknowledge the initial encouragement to write this text by 
my former associates at the Directorate of Management Analysis of the 
Department of the Air Force and to thank Harold Fassberg, Walter W. 
Jacobs, Robert R. Meyer, Paul Rech, Thomas L. Saaty, and Kenneth 
Webb for their many valuable suggestions. Special appreciation is due 
Mrs. Thelma Chesley and Mrs. Anne Bache for their typing of the past 
editions and Miss Joanne Wagner for her excellent typing of the current- 
edition manuscript. 


Saul I. Gass 
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LINEAR 
PROGRAMMING 


PART ONE / INTRODUCTION 


CHAPTER 1 / GENERAL DISCUSSION 


1. LINEAR-PROGRAMMING PROBLEMS 


Programming problems are concerned with the efficient use or allocation 
of limited resources to meet desired objectives. "These problems are char- 
acterized by the large number of solutions that satisfy the basie conditions 
of each problem. The selection of a particular solution as the best solu- 
tion to a problem depends on some aim or over-all objective that is 
implied in the statement of the problem. A solution that satisfies both the 
conditions of the problem and the given objective is termed an optimum 
solution. A typical example is that of the manufacturer who must deter- 
mine what combination of his available resources will enable him to 
manufacture his products in à way which not only satisfies his production 
schedule, but also maximizes his profit. "This problem has as its basic 
conditions the limitations of the available resources and the requirements 
of the production schedule, and as its objective the desire of the manu- 
facturer to maximize his gain. 

We shall mainly consider only à very special subclass of programming 
problems called Unear-programming problems. A linear-programming 
problem differs from the general variety in that a mathematical model or 
description of the problem can be stated, using relationships which are 
called "straight-line," or linear. Mathematically, these relationships are 
of the form 


gio coude epa petenda cT 


where the a;'s and b are known coefficients and the z;'s are unknown varia- 
bles. The complete mathematical statement of a linear-programming 
problem includes a set of simultaneous linear equations which represent 


1 Geometrically, these relationships are equivalent to straight lines in two dimen- 
sions, planes in three dimensions, and hyperplanes in higher dimensions. 
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the conditions of the problem and a linear function which expresses the 
objective of the problem. In Sec. 2 we shall state a number of program- 
ming problems and formulate them as linear-programming problems. 

To solve a linear-programming problem, we must initially concern 
ourselves with the solution of the associated set of linear equations. 
There are various criteria which can be applied to a set of linear equations 
to reveal whether a solution or solutions to the problem exist (see Dickson 
[34])..| The set of two equations in two variables 


2zi 4 3234 — 8 
314-2235 
has the unzque solution z? — 1 and x» — 2, while the single equation 
zxi-d- 2273 —- 8 (1.1) 
has an znfinite number of solutions. From (1.1) we have 
2; 8 — 2x3 Or X» — 4 — lox 


For every value of x» (or zi) there is a corresponding value of z, (or 23). 
If we further restrict the variables to be nonnegative, that is, zj 2 0 and 
z» 2 0, we limit the range of the variables, since 


2128—225220 implies ()excup E 
and 
434—54—1l$23,20 implies (Ep 


We still have an infinite number of solutions, but the addition of further 
restrictions or constraints to (1.1) has resulted in less freedom of action. 
As we shall show, the condition of nonnegativity of the variables is an 
important requirement of linear-programming problems. Systems like 
(1.1) in which there are more variables than equations are called under- 
determined. In general, underdetermined systems of linear equations 
have either no solution or an infinite number of solutions. 

One important method of determining solutions to underdetermined 
systems of equations is to reduce the system to a set containing just as 
many variables as equations, i.e., a determined set. "This can be accom- 
plished by letting the appropriate number of variables equal zero. For 
example, the underdetermined system 


221 92» d- 234 — 8 


pu 
Zi 2x3 4-23 — 5 bo) 


! Numbers in brackets refer to the publications listed in the References at the 
back of the book. 
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has three such solutions: 


m-0  m-!U( me--M 

qi 1i$ ap eem (QU) dU nA 
and 

mores 14 —2 t; — Of 


Mathematically, linear programming deals with nonnegattive solutions 
to underdetermined systems of linear equations. As we shall show in the 
succeeding chapters, the only solutions we have to be concerned with are 
those corresponding to determined subsets of equations that have been 
obtained in the manner described above. If, for example, we let Eqs. 
(1.2) represent the conditions of a linear-programming problem, we need 
only to consider the two nonnegative solutions z; — 1, 3 2 2, 233 — 0 
and z, — 1l4, x; — 0, z; — 24. "The remaining solutions fail to satisfy 
either the nonnegativity requirements or other criteria to be discussed. 

Just as the general programming problem has some objective that 
guides the selection of the solution to be used, the linear-programming 
problem has a linear function of the variables to aid in choosing a solution 
to the problem. "This linear combination of the variables, called the 
objective function, must be optimized by the selected solution. If for 
(1.2) we wished to maximize the objective function z; 4- x» 4- x5, then, 
of the two nonnegative solutions, the solution z; — 114, x» — 0, x; — 24 
is the optimum, as it yields a value of 134 for the objective function com- 
pared with a value of 3 for the other nonnegative solution. If we wanted 
to minimize the objective function z; — x», then the solution xz, — 1, 
X» — 2,13 — 0 would be the optimum, with a value of —1. As we have 
implied, the optimum solution either maximizes or minimizes some linear 
combination of the variables. Since the maximum of a linear function is 
equal to minus the minimum of the negative of the linear function, we lose 
no generality by considering only the minimization problem. 

With the added condition of optimizing an objective function, we are 
now able to select a single solution that satisfies all the conditions of the 
problem. "There might be multiple solutions in that more than one non- 
negative solution to the equations gives the same optimum value of the 
objeetive function. Generally speaking, combining the linear constraints 
of the programming problem with the optimization of a linear objective 
function transforms an underdetermined system of linear equations that 


1 There are, of course, an infinite number of other solutions to (1.2), which can be 
obtained by arbitrarily setting one of the variables equal to a constant; e.g., with 
z; — a we have z; — (11 — 3a)/4 and zs — (—1 -- a)/4. 
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describes a programming problem with many possible solutions to à 
system that can be solved for a solution that yields the unique optimum 
value of the objective function. 

We next give the general mathematical statement of the linear- 
programming problem: 

Minimize the objective function 


mice oRemd 527688 


subject to the conditions 


X1 Es 0 
ND) 2 0 
X; P 0 
£42 0 
and 
Qury o32s-- ^to Q3;1; gets oc up. c Dy 
03134 d 84353 "MF ^ oce ox Q5;3; de cot soR daada tm bs 
Qiii T GioX2 4 i ME Qi; "E s QinXaà — bi 
O11 ns Qm 2X» 4 sias zs Qj sts DE T Omni, — b 
where thec;forj — 1,2, . . . , n; b;for? — 1,2, . . . , m;and a;; are all 


constants, and m « mn. The c; are called cost coefficients. 


As will be discussed in the following chapters, every linear-program- 
ming problem has either: 


1. No solution in terms of nonnegative values of the variables 
2. A nonnegative solution that yields an infinite value to the objective 
function 


3. A nonnegative solution that yields a finite value to the objective 
function 


À linear-programming problem that describes a valid, practical program- 
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ming problem usually has a nonnegative solution with a corresponding 
finite value of the objective function. 


2. EXAMPLES OF LINEAR-PROGRAMMING PROBLEMS 


To illustrate the application of the above mathematieal description of the 
linear-programming model, we shall next discuss the linear-programming 
formulation of three problems. A more detailed discussion of these and 
other problems is given in Part 3. 

The transportation problem. | manufacturer wishes to ship a num- 
ber of units of an item from several warehouses to a number of retail 
stores. Each store requires a certain number of units of the item, while 
each warehouse can supply up to a certain amount. Let us define the 
following: 


m -— the number of warehouses 

n — the number of stores 

à; — the total amount of the item available for shipment at ware- 
house 2 

b, — the total requirement of the item by store 7 

z;; — the amount of the item shipped from warehouse 2 to store 7 


We shall assume that the total amount available 1s equal to the total 
required, that is, p Q; es Y bj. As will be shown later, this assumption 
i Jj 


is not a restrictive one. 
'The z;; are the unknown shipments to be determined. If we form 
the array (for m — 2 and n — 3) 


Stores 


Warehouses 


we see that the total amount shipped from warehouse 1 can be expressed 
by the linear equation 


Q1), 2132 "b Tas 7 0a (2.1) 
For warehouse 2, we have 


q21 d- X2» d- Xs — aw (2:2) 
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We also note that the total amounts shipped to the three stores are 
expressed by the equations 


E£ucbxno—b 

Y1» d X3» — b» (2.3) 

Xi3 Fo X23 — 53 
'The manufacturer knows the cost c;; of shipping one unit of the item from 
warehouse 2 to store j. We have the additional assumption that the cost 
relationship is linear; i.e., the cost of shipping z;; units 18 cijzi;. 

' The manufacturer wishes to determine how many units should be 
sent from each warehouse to each store so that the total shipping cost is à 


minimum. This objective of minimizing the cost is achieved by minimiz- 
ing the linear cost function 


Citi d- Citi» dF- CisdXis Ro CoiY2i dB CorX2» de Costo3 (2.4) 


Since a negative z;; would represent a shipment from store j to warehouse 
i, we require that all the variables ri; 2 O0. 


By combining Eqs. (2.1) to (2.3), the objective function (2.4), and 
the condition of nonnegativity of the variables, the transportation prob- 
lem for m — 2 and n — 3 can be formulated in terms of the following 
linear-programming problem: 

Minimize the cost function 

Ci1X11 d. Cista2 - Civis - CoiX21 d- Cos322 d- Cos$53 


subject to the conditions 


Q2 eu 


and 
iic £id- xx — Q4 
Xi dRoY:9 X23 — Q3 
Tu "Ia — bh 
X12 JP 32s zb. 


T13 doxaa — 53 
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Activity-analysis problem. A manufacturer has at his disposal fixed 
amounts of à number of different resources. "These resources, such as 
raw material, labor, and equipment, can be combined to produce any 
one of several different commodities or combinations of commodities. 
The manufacturer knows how much of resource 2 it takes to produce one 
unit of commodity j. He also knows how much profit he makes for each 
unit of commodity j produced. "The manufacturer desires to produce 
that combination of commodities which will maximize the total profit. 
For this problem, we define the following: 


m -— the number of resources 

5^ — the number of commodities 

G;; — the number of units of resource ? required to produce one unit of 
commodity j 

b; — the maximum number of units of resource 2 available 

c; — profit per unit of commodity j produced 

xz; — the level of activity (the amount produced) of the jth commodity 


'The a;; are sometimes called input-output coefficients. 
'The total amount of the zth resource that 1s used 1s given by the linear 
expression 


Our) cM EIL. 


Since this total amount must be less than or equal to the maximum num- 
ber of units of the 7th resource available, we then have, for each 7, à 
linear inequality of the form 


Gv dx.cp Ue c6 ns S b; 


As a negative x; has no appropriate interpretation, we require that 
allz; 2» 0. The profit derived from producing z; units of the jth com- 
modity is given by cjzj Our formulation has yielded the problem of 
maximizing the profit function 


exty b Cu2an de ^ "05 Can 
subject to the conditions 


T1 20 
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and 
Xi - diva coco 0 B 8434Xa SS Di 
Q2iXi 4e Gata dB ot ^ * B oO3sXa X bs 
QOm1X1 T Qn 2X2 T ab 4s -F Om nX n E D» 


Since, as will be discussed later, an inequality has an equivalent repre- 
sentation as an equality in nonnegative variables, the above problem is 
another statement of the general linear-programming problem. 

The diet problem. | Here we are given the nutrient content of a num- 
ber of different foods. For example, we might know how many milli- 
grams of phosphorus or iron are contained in 1 oz of each food being 
considered. "We are also given the minimum daily requirement for each 
nutrient. Since we know the cost per ounce of food, the problem 1s to 
determine the diet that satisfies the minimum daily requirements and is 
also the minimum-cost diet. Define 


m -— the number of nutrients 

^" — the number of foods 

aj; —^ the number of milligrams of the :th nutrient in 1 oz of the jth food 
b; — the minimum number of milligrams of the ?th nutrient required 

c; — the cost per ounce of the jth food 

z; — the number of ounces of the jth food to be purchased (v; 7 0) 


The total amount of the ?th nutrient contained in all the purchased 
food is given by 


GaXxi-F 0aXrcd o7 oBasm, 


Since this total amount must be greater than or equal to the minimum 
daily requirement of the ith nutrient, this linear-programming problem 
involves minimizing the cost function 


£124 9-600495 "EO, 
subject to the conditions 


X1 zu) 
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and 
03114 7-503245 PF * t3 disi ENIM 
0123 -d- 03:23 -d- * * * -LB ü3nds 2t b» 
Gm "HE Gm?» d rr s uis GmnXn zt b, 


3. SURVEY OF LINEAR-PROGRAMMING APPLICATIONS 


The early applications of linear-programming methods fell into three 
major categories: military applications generated by the Air Force's 
Projeet SCOOP, interindustry economies based on the Leontief input- 
output model, and problems involving the relationship between zero- 
sum two-person games and linear programming.! In the past few years 
these areas of application have been extended and developed, but the 
emphasis in linear-programming applications has shifted to the industrial 
area. In this section we shall describe in general terms many of these 
applications, so as to give the reader an idea of the adaptability and 
suecess of the linear-programming model. Although the following 
material was taken from case studies, we shall not cite specific references 
in the body of the text. Instead we have grouped in a more readable 
fashion the corresponding selected set of references in the Bibliography. 
Additional references are cited in appropriate places in the text. Fora 
more complete list of applications the reader is referred to the bibliography 
by Riley and Gass [87]. In this bibliography, the authors have compiled 
linear-programming applications into a number of classifications. We 
shall follow their arrangement in our discussion. 

a. Agricultural applications. 'Tlhese applications fall into two cate- 
gories, farm economies and farm management. "The former deals with 
agricultural economies in the large, i.e., as related to the economy of à 
nation or region, while the latter is concerned with the problems of the 
individual farm. 

One study in farm economies deals with interregional competition 
and the optimum spatial allocation of erop production in the United 
States. Efficient production patterns were specified by a linear-program- 
ming model constrained by regional land resources and national demands. 
The models used were based on 122 producing regions and included as 
many as 500 constraints, including upper bounds on each crop category 
within regions. Production patterns were indicated to allow minimum 
national food costs and alternatives in livestock feed substitution. "Three 


! For purposes of exposition we have kept the discussion of linear programming 
and the theory of games for Chap. 12. 
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models were used and a set of national supply prices for erops was derived 
for each. The quantitative analysis suggests the amount of land to be 
withdrawn from crop production and shifted to less intensive uses. 

An application of linear-programming techniques to a typical farm- 
management problem is that of allocating limited resources such as 
acreage, labor, water supply, working capital, etc., in such a way as to 
maximize net revenue. "The problem is to choose simultaneously the 
partieular erop or crops to be grown in the following period, the number 
of acres of land to allocate to each of these crops, and the particular 
method to use in the production of each of these crops so that net cash 
return will be maximized. Another linear model of a more general 
nature considers the problem of the selection of a crop-rotation plan by 
an individual farmer. "This applieation has been developed for both 
the static and dynamic situations. 

As described in Chap. 11, one of the first applications to farm- 
management problems was the minimum-cost feed problem. 

b. Contract awards. ln addition to the procurement problem 
described in Chap. 10, the linear-programming model has been applied to 
competitive bond bidding. Competitive bidding for serial bonds issued 
by governments and other public authorities is based on the net-interest- 
cost method. "The bidder presenting the lowest net-interest cost to the 
issuing authority wins the issue. The linear-programming model con- 
siders the factors which enter into the net-interest cost and provides a 
method for adjusting those variables most subject to the control of the 
bond bidders. The model which arises from the minimization require- 
ment admits an explicit solution. 

c. Industrial applications. cHEMICAL INDUSTRY. The applications 
to the chemical industry have been mostly of the production and inven- 
tory-control type. One study was made to find the optimal scheduling 
of 25 machines of varying capacities used in electrochemical formation of 
aluminum oxide films on etched aluminum foil. "The foil comes in about 
45 combinations of voltage capacity and width, each with its correspond- 
ing current requirements. "The total current load is restricted by limita- 
tions of the company's power-distribution facilities. "This scheduling 
problem turns out to be a straightforward linear-programming problem 
which involves the capacities, power limitations, and other restrictions. 

THE COAL INDUSTRY. .À model for the coal industry has been formu- 
lated which is eomposed of two interrelated linear-programming problems. 
The data of the model are spatially distributed demands for coal and the 
unit costs of deliveries from the deposits to the demand locations. 'The 
levels of the deliveries are the variables of the first programming problem. 
They are selected to minimize the cost of meeting the demands subject 
to the eapacity restrictions for the coal deposits. "The variables of the 
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second programming problem are the delivered prices of coal at the 
demand locations and the unit royalties earned by the various deposits. 
The values of these variables are selected to maximize total revenue net 
of royalty payments. 

COMMERCIAL AIRLINES. "The work done in this area has dealt with 
the problems of routing aircraft and of airline management. One study 
used linear-programming techniques in determining the pattern and 
timing of flights in the Scottish region of British European Airways. 
Only a small number of aireraft and crews operate there, but relatively 
large losses occur. Optimal operating policy was first determined, and 
then a static model was formulated describing point-to-point flying as 
activities with linear input and output coefficients. Inputs are aircraft, 
aircrews, and money, while outputs are aircraft, aircrews, and services. 
'Time of day may be added for a dynamic extension of the model. 

COMMUNICATIONS INDUSTRY. "The main work has been in the optimal 
design and utilization of communication networks. Problems involving 
facilities for transmission, switching, relaying, etc., have been solved by 
linear-programming methods. "These methods provide a general system 
approach for resolving the complex interactions among system capacities, 
users! demands, and economic factors. 

IRON AND STEEL INDUSTRY. 4À number of models dealing with pro- 
duction planning in the iron and steel industry have been formulated. 
One such study derives a minimum-cost steel production program from a 
linear-programming model which is used to determine the optimum 
monthly plan of the open-hearth shop, the rate of hot-metal production 
by the blast-furnace department, and the amount and type of steel scrap 
to be purchased. "The monthly production plan becomes a function of 
the demand for steel by major types of steel, the amount and type of the 
available mill serap, and the price and availability of several types of 
scrap on the open market. 

A report of a linear-programming study made in the tube mill of à 
large manufacturer showed that scheduling for maximized profit would 
increase company profits by close to $350,000 over the previous year. On 
the other hand, scheduling for maximized output would increase the quan- 
tity by 22 per cent, but profits would decrease 23 per cent, or about 
$300,000. "The increase in profit was the direct result of a selection of 
produets using linear programming. Factors considered are plant and 
machine capacity, sales forecasts by various warehouses, shipping rates 
by items to the various distribution centers, and present and proposed 
company policies. 

PAPER INDUSTRY. "Two applications of linear programming in the 
pulp and paper industry have been in the transportation problem and in 
trim scheduling. "Transportation scheduling deals with the problem con- 
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fronting a company having several mills. "The problem is how to assign 
the various orders to the mills so as to reduce the total company freight 
bill to a minimum. . Trim scheduling deals with the matter of reducing 
trim waste on the paper-cutting machine. "The manufacturer makes rolls 
of newsprint paper to meet customers' specifications as to width and 
diameter. In cutting these customer rolls from larger rolls of paper, 
trim losses areincurred. Here the manufacturer must determine on which 
machines and in what combinations the orders should be cut to result in 
a minimum over-all trim loss. An approach of this kind was successful 
in showing the way to an increase in trim efficiency of over 1.5 per cent. 
This is equivalent to an increase in production of over 15 tons per day. 

PETROLEUM INDUSTRY. This industrial area has furnished a great 
many important and interesting linear-programming applieations. "The 
earliest of these was the problem of blending gasolines into required 
products for maximum profit. "The final products must meet a variety of 
specifications, e.g., octane number and volatility, in such a manner as to 
maximize the net receipts. Other studies include the problems of opti- 
mum erude allocation to several refineries, and the optimum inventory 
and production rate for à seasonal product. Mathematical models of 
refinery operations and of the petroleum industry in general have led 
to the study and solution of many nonlinear programming problems. 

RAILROAD INDUSTRY. À linear-programming model for optimal pro- 
gramming of railway freight-train movements has been formulated to 
handle scheduling problems as found at a large terminal-switching railroad. 
The constraints of the model were based on the methods of hiring and 
paying the trainmen, the scheduling of shipments, and the capacity 
limitations of the railroad, the objective being to minimize the total crew 
and engine expense. 

Other railroad applications have dealt with the distribution of freight 
cars and the distribution of classification effort between yards. 

d. Economic analysis. "The use of linear-programming techniques in 
the field of economies has not been limited to the Leontief interindustry 
model. Anotherimportant application has been the linear-programming 
interpretation of the theory of the firm. Here we deal with the problem 
of finding the production program which will make the profits of the firm 
as great as possible, with the limitation that the program must not 
require more than the total available supply of any resource. 

The problem of selecting an investment portfolio has also been treated 
by programming techniques. Here we start with the relevant beliefs 
about the securities involved and end with the selection of a portfolio. 

In addition to diet, many problems in the general area of marketing 
analysis have been formulated as linear-programming problems. One 
such model was constructed to describe the behavior of customers in a 
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food market, based upon experimental data resulting from a survey by 
the British. Ministry of Foods of approximately one thousand urban 
working-class households. "These data served to provide average dis- 
tribution of food purchases and expenditures for the sample as a whole, 
and of the quantities of nutritional factors derived from the diet. For 
purposes of the analyses, the foods were classified into 15 groups and the 
number of nutritional faetors considered was 12. The expenditures for 
each of four diets meeting a specified set of requirements were computed 
and compared with the results of the survey. 

Special cases of the theory of plant location, i.e., the selection of sites 
for factories or branch stores so as to maximize profits, have been investi- 
gated by linear-programming techniques and the procedures of activity 
analysis. One example considers the reaction of competitors and formu- 
lates the problem as an equivalent two-person zero-sum game. 

Àn unusual experiment involving the methods of linear programming 
was designed to measure the cardinal utility of nonmonetary outcomes 
and to use the computed utilities to predict further choices. Here seven 
music students were used as subjects with long-playing records as the 
outcomes. "The students individually attended two sessions, and a cor- 
responding joint utility curve was developed. The joint curve was used 
to predict choices that would be made among new combinations of records, 
and these predictions were tested in a third session. It was found that 
the predictions made by the linear-programming model were somewhat 
more accurate than the predictions made by the usual method. 

e. Military applications. One of the earliest linear models was that 
of the air-lift problem. Here the constraints involved the supplies to 
Berlin, number of runways available, number of crews and aircraft, and 
money available. "The objective was either to deliver a specified number 
of tons at à minimum cost or to maximize tonnage supplied with a given 
supply of aircraft and money. 

Another Air Force application, the aircraft-deployment problem, is 
concerned with the efficient allocation of limited resources such as combat 
aircraft and trained erews. Here we are given the numbers of aircraft 
to be produced in the successive months of à program. These aireraft 
are either deployed (sent into a combat area) or else used to train crews. 
Aircraft are deployed only when there are crews available to man them, 
and once deployed, they remain at the assigned station for the duration of 
the program. Aircraft sent to training may be deployed after a given 
number of months. Each aircraft in training produces a specified number 
of new crews each month. "The object is to allocate the new aircraft 
between the combat and training missions so as to maximize the total 
number of aircraft-months of deployed aireraft. 

The caterer problem (see Exercise 2 of Chap. 11) is a paraphrased 


16 INTRODUCTION [Chap. 1 


version of a military problem which arose in connection with the estima- 
tion of aireraft spare-engine (i.e., napkins) requirements. 

Other military examples include the problem of selecting an air 
weapon system against guerillas so as to keep them pinned down and at 
the same time minimize the amount of aviation gasoline used; a variation 
of the transportation problem that maximizes the total tonnage of bombs 
dropped on a set of targets; and the problem of community defense 
against disaster, the solution of which yields the number of defense units 
that should be used in a given attack in order to provide the required 
level of protection at the lowest possible cost. 

f. Personnel assignment. 'The general statement of the personnel- 
assignment problem is given in Chap. 10. A special dynamie assign- 
ment problem considers the assignment of toll collectors so as to man the 
required number of toll booths with the minimum number of men for 
given time periods. 

g. Production scheduling and inventory control. | & basic formulation 
of the production-scheduling and inventory-control problem is discussed 
in Chap. 11. There we consider the problem of smoothing production to 
meet stipulated requirements in a manner that minimizes storage costs. 
An earlier linear-programming treatment of a similar problem considers 
the warehouse problem. Here we wish to determine the optimum use of 
a storage warehouse for commodities subject to price fluctuations. This 
problem has since been generalized to include multiple products and ware- 
houses, and varying prices. 

À problem that has been investigated in many forms by linear-pro- 
gramming techniques 1s that of assembly-line balancing. "The basic 
problem is as follows: The commodity to be assembled is composed of 
many different components. Itisnecessary to assemble these component 
parts in some specified sequence or set of sequences. Each assembly 
operator must be assigned a number and combination of parts to assemble 
such that the sum of the times required to carry out his assigned task is 
equal to or less than the cycle time, i.e., the amount of time elapsing 
between successive units as they move along the assembly line. An 
operator has idle time if the work time assigned is less than the cycle time. 
The assignment of jobs to each operator is to be done so as to minimize 
his idle time. A variation of this problem is the multistage production 
line. Here we have a number of items which must go through one produc- 
tion stage or machine and then a second, ete. At most, one item can be 
on a machine at a given time. For each item we know the setup time 
plus work time at each stage. "The problem is to schedule these items 
through the production line so as to minimize the total elapsed time. 

Other applications include the problem of determining the number of 
each type of an item to be produced by each routing in a machine-shop 
production line so that the total production cost is minimum and delivery 
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time and requirements are satisfied using existing facilities; and the 
probleim of procuring and assigning new transport aircraft to the various 
transport jobs in order to minimize the cumulative operations-plus- 
procurement costs. 

h. Structural design. | Problems in this area involve the linearization 
of engineering principles related to the theory of plastie collapse and 
structural design. The eritical value of the loading parameter at which 
plastie collapse occurs in structures for certain types of loading has been 
characterized by means of a maximum or minimum principle. "The maxi- 
mum principle can be reduced in various ways to the problem of maximiz- 
ing a linear function subject to linear inequalities. 

The problem of designing plane frames in such à way that the 
material consumption 1s a minimum can also be formulated as a linear 
model. 

7. Traffic analysis. 'This application deals with the problem of 
scheduling traffic signals. The mathematical formulation of a street- 
network system assumes knowledge of the following parameters: total 
traffie-signal eycle (red plus green), the fraction of the cycle that is red 
at each intersection, and the maximum number of vehicles that can move 
through the intersection in each direction. "The model can handle such 
phenomena as variations in average speed along different portions of the 
route, turn-ons and turnoffs, variation in traffic capacity with intersection 
and flow direction, the capacity of blocks for holding stopped vehicles, 
three-way lights or other special light schedules, delays in starting up after 
the light turns green, and the appearance of random delays due to causes 
other than lights. "The eriterion for obtaining an optimum time phasing 
of the lights is that the number of delays be minimized. 

j. Transportation problems and network theory. Discussions of the 
transportation problem and some of its variations are given in Chap. 10. 
The mathematical model of the problem has proved extremely versatile 
and susceptible to a number of computational procedures. 

An associated problem is that of maximal flows in networks (see 
Chap. 11). The problem is the following: Consider a network (e.g., rail, 
road, communications) connecting two given points by way of a number 
of intermediate points, where each link of the network has a number 
assigned to it representing its capacity. Assuming a steady-state condi- 
tion, find a maximal flow from one given point to the other. A simple 
computational method, based on the simplex method, has been developed 
for solving this problem. 

An extension of the basic transportation problem is that of the trans- 
shipment problem. Here any shipping or receiving point is also per- 
mitted to act as an intermediate point in seeking an optimum solution. 

k. Traveling-salesman problem. 'The problem is to find the shortest 
route for a salesman starting from a given city, visiting each city in a speci- 
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fied group, and then returning to the original point of departure. A 
computational procedure for the general problem has not been devised, 
but specific examples have been successfully solved by linear-program- 
ming techniques. In particular, the problem of touring 49 cities, one in 
each of the48 mainland states other than Alaska and one in the District of 
Columbia, has been formulated as a linear-programming model and solved. 

l. Other applications. Additional applications of linear programming 
include the efficient operation of a system of dams and the design of 
optical filters. "The first problem is to determine variations in the stor- 
ages in six Missouri River dams so as to maximize the energy obtained 
from the entire system and thereby increase the efficiency of operation. 
Physical limitations on the storages and releases appear in the form of 
inequalities. An approximation to a change in energy is given as a linear 
function of the variations. 

In the second problem, products of exponentials are converted to 
sums of the logarithms to provide a linear-programming approach to the 
synthesis of filter combinations for optimum fit of combination filters to 
a desired spectral-transmission curve. "The unknowns to be determined 
could be either the concentrations of various possible filter constituents, 
which are to be mixed to form a single filter, or the thicknesses of various 
filters which are to be placed in series. 

In addition to their wide application to physieal situations, linear- 
programming techniques have been firmly related to a number of theo- 
retical areas of mathematies. Some of these disciplines have lent their 
power to the solution of special types of linear-programming problems; 
e.g., the theory of statistics has been applied in solving problems whose 
elements are subject to random errors. What is more important, how- 
ever, has been the use of linear-programming methods in the proof of 
theorems and solution of problems in what appear to be totally unrelated 
fields of investigation. The theoretical and computational aspects of 
linear programming have been applied with much success to the areas of 
combinatorial analysis, partially ordered sets, network and graph theory, 
and systems of distinct representatives. Since descriptions of selected 
applieations from these areas would entail introducing material which is 
beyond the scope and purposes of this book, we leave to the interested 
reader the investigation of the specific details of these mathematical 
applieations of linear programming. | Appropriate references are given in 
Sec. 12 of the Bibliography. 


REMARKS 


For additional introductory material, the reader is referred to Dorfman [38], 
Cooper and Charnes [16], Henderson and Schlaifer [55], and Directorate of Manage- 
ment Analysis [36]. 
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1. Write out the complete linear-programming formulation of the following trans- 
portation problem and attempt to determine the optimum solution by trial-and-error 


methods: 


À manufacturer has distribution centers located at Atlanta, Chieago, and New 
These centers have available 40, 20, and 40 units of his product, respec- 


York City. 


tively. 


His retail outlets require the following number of units: Cleveland, 25; 


Louisville, 10; Memphis, 20; Pittsburgh, 30; and Richmond, 15. "The shipping cost 


per unit in dollars between each center and outlet is given in the following table: 


Atlanta... 
Chicago. . 
New York 


Cleveland | Louisville | Memphis | Pittsburgh | Richmond 
55 30 40 50 40 
35 30 100 45 60 
40 60 95 35 30 


2. A furniture manufacturer wishes to determine how many tables, chairs, desks, 
or bookeases he should make in order to optimize the use of his available resources. 
"These products utilize two different types of lumber, and he has on hand 1,500 board 
feet of the first type and 1,000 board feet of the second. He has 800 man-hours 
available for the total job. His sales forecast plus his back orders require him to 
make at least 40 tables, 130 chairs, 30 desks, and no more than 10 bookceases. Each 
table, chair, desk, and bookease requires 5, 1, 9, and 12 board feet, respectively, of 
the first type of lumber and 2, 3, 4, and 1 board feet of the second type. A table 
requires 3 man-hours to make, a chair 2, a desk 5, and a bookease 10. The manu- 
facturer makes a total of $12 profit on a table, $5 on a chair, $15 on a desk, and $10 on 
a bookcase. Write out the complete linear-programming formulation of this problem 
in terms of maximizing the profit. 

3. A baker starts the day with a certain supply of flour, shortening, eggs, sugar, 
milk, and yeast. He specializes in making bread, cakes, English muffins, and cookies. 
He wishes to determine how much of each product he should make so as to maximize 
his profit. "The recipes are given in the following table (we ignore such plentiful sup- 
piies as salt, water, etc.). 


Available 

Bread Cake Muffins Cookies | resources 
IBIQU ES EC ERIT 1293 3C 96 C 36 C b C 
Shortening....... 2 tbsp 12 tbsp 3 tbsp 4 tbsp b, tbsp 
Eggs 0 3 1 1 bs 
SUEgAR EAE 4C 36 C 1$ C 1€ 6,C 
Milk 25) 34 C il e 0 bs; C 
CASU D EEUU ERE URU 1l cake 0 1 cake 0 b, cakes 
«Profi e UE IE TOUS €1 C2 C3 C4 
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Write the corresponding linear-programming formulation and discuss the applicability 
of such a model to a real-life situation. 

4. Describe the model of a transportation problem in which the total availabilities 
are greater than the total requirements. From the mathematical point of view, why 
must the sum of the availabilities and the sum of the requirements be equal in the 
standard transportation model? Discuss the linearity assumption of the transporta- 
tion-problem cost relationship. 

5. Formulate the following diet problem. | A mother wishes her children to obtain 
certain amounts of nutrients from their breakfast cereals. "The children have the 
cholce of eating Krunchies or Crispies or a mixture of the two. | From their breakfast 
they should obtain at least 1 mg of thiamine, 5 mg of niacin, and 400 cal. One ounce 
of Krunchies contains 0.10 mg of thiamine, 1 mg of niacin, and 110 cal; 1 oz of Crispies 
contains 0.25 mg of thiamine, 0.25 mg of niacin, and 120 cal. One ounce of Krunchies 
costs 3.8 cents and 1 oz. of Crispies 4.2 cents. 


CHAPTER 2 / MATHEMATICAL 
BACKGROUND 


A complete development and understanding of the theoretical and com- 
putational aspects of linear programming requires the blending of the 
basie concepts and techniques of à number of mathematical topies. In 
this chapter we shall present and discuss only those elements of these 
topies that facilitate the discussion, are required in the succeeding chap- 
ters, or will aid the reader in applying the material to be described. 


1. MATRICES AND DETERMINANTS 


À maírix is a rectangular array of mn numbers arranged in m rows and 
n columns as follows: 


O11  Q12 G1n 
Q»1  Q22 O25 
Gmi QOm2 ^" "^ ' Qmm 


Such an array is usually enclosed in parentheses and is called matrix A. 
'The individual a;; are called elements. We sometimes denote the matrix 
A by (a5). For any m or n and any elements a;; we have 


Q11 012 Q1n 
(17100 0/0» MEC) 

ADM Me Rr .]7 «2 
Qmi GQm2 ^" ' '" QOmn 


Matrix À is called square if m. — n and is said to be of order n. 

À. column vector is à matrix with only one column, and a row vector 
is à matrix with only one row. A d7agonal matrix is a square matrix 
whose off-diagonal elements, a;; for ? 7 j, are all equal to zero. 

À unit matrix, or identity matrix, is à diagonal matrix whose diagonal 


22 INTRODUCTION [Chap. 2 


elements are 1l. A unit matrix of order n is denoted by L, or simply I. 
For n — 8, we have 


i 0 0 
I; (0 1 O 
g uu 


The transposed matrix, A', of a matrix A is 


Q11 02 Om1 

Q12  Q22 Om 2 
A' z 

Q15 Q2n its Omn 


i.e., the rows and columns are interchanged. 
Two matrices are equal if and only if their corresponding elements are 
equal. Hence the corresponding dimensions, m and n, must also be equal. 
À square matrix À is (riangular iM all aj; — 0 for? » j orif alla;; 2 0 
for? « j. "The matrix 


Qi O12 GOG13 Q4 
0 Q29 23  Q24 
0 0 Qaa  Q34 
0 0 0 (44 


is triangular. 

À matrix À is symmetricif À — A', thatis,if a; — aj. A matrix ls 
skew-symmetric iM A — —A', that 1s, if aj; — —aj. "This last condition 
implies that all aj; — 0 for 2 — j. 

À null matrix has all its elements equal to zero and is denoted by 
O — (0). Inthe succeeding chapters the matrix O will be used to denote 
either a null column vector or a null row vector. 

We next define the operations of matrix addition and multiplication. 

Given any scalar o (a real number a) and any matrix A, the product 
aÀ is given by 


QÜUmi  QGÜUm2 ^" ' ' Qmm 


Given any two m X n matrices À and B, the sum A -- B - C is 
given by 


Qii  Q312 Q1 bu 3o bis 
fei em ase er eto uon cod Hu a M ME NOE 
Qmi  Qm2 SN CSS neun b, bo ax E ban 
1j d 55 aya Dia. 0 0 odas sls bis 
zu rmm. MM MM C e (ai e bij) 


Om 1 SE bs Q2 X boo C — Omn d [0 
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The properties of scalar multiplieation and matrix addition are: 


(a) (A -B)--C-—-A (B4 C) (Associative law) 


(b) A--B-B-A (Commutative law) 
(c) c 8)À — oA -- BÀ 
a 5 j (Distributive law) 
a(AÀ 4- B) 2 oA 4- aB 
(d) A--O-A 


where A, B, and C are all of dimensions m X n and « and 8 are scalars. 

The multiplieation of two matrices A and B is defined only on the 
assumption that the number of columns of A is equal to the number of 
rows of B. On this assumption, the elements of the product AB — C are 
defined as follows: 

The element in the zth row and jth column of matrix C is equal to the 
sum of the products of the elements of the ?th row of A multiplied by the 
corresponding elements of the jth column of B. For example, 


a a a bn bna C C 
11 12 13 11 12 
AB — bo 5» E -€C 
Q»1 O22  Q23 b b €21 C22 
31 32 
where 


(pres abi; xg Gb»; 3 Gisbs; 


The product of a matrix À of dimensions m X n and a matrix B of 
dimensions n X q yields a matrix C of dimensions m X q. "The reader 
can easily verify that matrix multiplieation is not commutative; 1.e., 
in general AB z BA. 

Matrix multiplication has the following properties: 


(a) (AB)C — A(BC) (Associative law) 
(D ^ (A- B)C - AC 4- BC 
C(A -F B) —- CA 4- CB 
(c) a(AB) — («A)B — (A)(aB) 
(d) ATUS A 


(Distributive law) 


where matrices A, B, C, and I are of the correct dimensions and o is à 
scalar. We note that (AB)' — B'A'. 

Associated with every square matrix À is a number called the deter- 
minani of A. "The determinant of A, denoted by |A|, is obtained as the 
sum of all possible products in each of which there appears one and only 
one element from each row and each column of A, each product of this 
sum being assigned a plus or minus sign according to the following rule: 
Let the elements in a given product be joined in pairs by line segments 
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(see example below). If the total number of such segments sloping 
upward to the right is even, prefix a plus sign to the product; otherwise 
prefix a negative sign. (Each determinant has n! such products where 
n is the order of the matrix.!) 

For» — 3, we have 


A -—[|asn do / 02: 
Q31 Q3»  Q33 
and 
|au Q5 7 fs 
|A| — |Güs17 0232 / Q3 


Q31 O32 O33 


— 431022033 d- 012053031 d- 01321032 
— 0310233032 — 012021033 — 013022031 


The lines for determining the sign of the third term have been drawn in 
the array for the determinant. 

The following properties of determinants can be verified from the 
above definition: 


1. If every element of à column or of a row of a determinant 1s zero, 
then the value of the determinant is zero. 

2. The value of a determinant is not changed if corresponding columns 
and rows are interchanged. 

3. If |B| is the determinant formed by interehanging two columns or 
rows in |A|, then |B| 2 —/A|. 

4. If two columns or rows of a determinant are identical, then the 
determinant has a value of zero. 

b. If every element of a column or of a row of a determinant is multi- 
plied by à fixed number k, then the value of the determinant is multi- 
plied by k. 

6. The value of a determinant is not changed if, to every element of a 
column or row, we add k times the corresponding element of another 
column or row. 


The rank of any matrix A is the order of the largest square array in A 
whose determinant does not vanish, i.e., does not have a value of zero. 

A square matrix À is said to be nonsingular if its determinant does not 
equal zero. If |A| — 0, the matrix is singular. 


! The factorial notation of n! is defined as n! 2 n(n — 1)n —2) - - - 2.1 


where 0! 2 1 : 
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The minor D; of the element 4;;is the determinant obtained from the 
square matrix A by striking out the zth row and jth column. 

The cofactor A;; of the element a;; equals (—1)*D;.  Aiis referred 
to as the signed minor of the element a;;. 

The adjoint of an n X n matrix A is another n X n matrix J ^ (Aj) 
in which the element in the ?th row and jth column is the cofactor of the 
element in the jth row and ith column of A. We have 


Àj À5 9f A, 
meses 
Àis A», asi Ass 


A matrix Biscalled the znverse of thesquare matrix Aif AB — I. The 
inverse of À 1s denoted by A^!, For every nonsingular square matrix À 
there exists a unique À^! such that 


AA-! — A-!A —- I 
It ean be shown that, if |A| x 0, 


—-— 


T 
Sand 


We see that only nonsingular square matrices have inverses. We also 
note that (AB)! — B^!A-!. 


2. VECTORS AND VECTOR SPACES 


In the familiar two-dimensional Euclidean plane, points are represented 
by ordered pairs of numbers U — (ui,u;). U can be regarded as a point 
with coordinates (u,,u;) relative to a fixed origin O — (0,0) or as a vector, 
i.e., a translation of the origin from (0,0) by the amounts u; and u» in two 
fixed coordinate directions (Fig. 2.1). "These two notions are used inter- 
changeably. We shall sometimes write U — e 


uo» 


(u,,u2) 


u FIGURE 2.1 
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Let us call the two-dimensional Euclidean space E» and note the 
important properties of vectors in E». 

a. Multiplication of vectors by scalars (Fig. 2.2). To every pair a 
and U, where o is à scalar and U is a vector, there corresponds a vector 
called the scalar product of « and U, written oU — (euij,ou»), and 
such that 


(a) a(8BU) — (ag)U (Associative law) 
b U 4- V) 2» oU V 
M E x (Distributive law) 
(a 4- 8)U — aU -- 8U 
(c) IU -U 
(d) 0U O0 - (0,0) 
(au,,aus) 
(a»1) 


FIGURE 2.2 


b. Addition of vectors (Fig. 2.3). To every pair U — (uij,u;) and 
V — (viv;) of vectors in Es there corresponds a vector called the sum 
of U and V, written U 4- V — (ui; -F vi, us 4- vs) and such that 
(a) U-V-2V--U (Commutative law) 
(b) (UcT-V)-W-U-c-(V-c-W) (Associative law) 


FIGURE 2.3 
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There exists in E; a unique vector O, called the origin, such that 
(c) U-J-O-U for all U in E» 


To each U in E; there corresponds a unique vector, called the inverse of 
U and denoted by —U, such that 


( RNIUE. MID 0 


c. Inner product of vectors. "To every pair of vectors U and V in E» 
there corresponds a real number called the Znner product of U and V, 
written U - V — uq 4- us» and such that 


(a) U:V2zV.U (Commutative law) 
(b) (aU 4- 8V)  W 2 a(U- W) 4- 8(V - W) 
for all scalars «a and 8 and all vectors U, V, W in E». 


(c) DU 0 and ILU s if and only if U 2 O 


Two vectors U and V are said to be orthogonal to one another if 
U:.V-90O. 

d. Length of a vector (Fig. 2.4). 'To every vector U in E» there cor- 
responds a real number, called the length of U, written 


|U|| 2 4 vu? o us? 
and such that 
(a) |U| 20 | and  |U|-20  ifandonlyifU - O 
(b  ||eU| — le|- [[U]| 
The length satisfies the triangle inequality 
() | |U v VI S [UI] 4 IV 
for every U and V in E;. 
(d —|U|--- vU-U 


e. Distance between two vectors. "To every pair of vectors U and V in 
E» there corresponds a real number, called the distance between U and V, 


FIGURE 2.4 
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written d (U,V), such that 
d(U,V) eU — VJ e V (ui PESCE s ce 02) 


A set of vectors Uj, U, . . . , U; is called linearly independent if, for 
all numbers oi, o», . . . , €» 

ajUi d- a9Uz -- : * - 4E o,U, — O 
implies 

tDUoR-s Oro IT — H2 9410) 


Otherwise the set is linearly dependent. For example, the set of two 
vectors 


is linearly independent. "To show this, we write 


(9) *() - () 


oue een 
pom) 


Since in the last equation the corresponding elements in the two column 
vectors must be equal, we have oa» — 0 and hence o; — 0. In à similar 
fashion we can demonstrate that the set of vectors 


gue im n M and | U,- (1) 


is linearly dependent. Here we have 


«Q«()«()-Q) 2n 


a 1U4 4- o3U» 


Il 


or 


Or 
Q1 d [o4 E 0 
py 
We are then led to the conclusions that o3 — —oi1ando; — —o;. Hence 
for o, — o» — « and o; — —a, where « can be any number, Eq. (2.1) 
holds. 


This last example illustrates the property of E? which makes it two- 
dimensional: that there exist two linearly independent vectors [e.g., the 
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unit vectors (1,0) and (0,1)], whereas every set of three vectors is linearly 
dependent. Generalizing this notion of dimensionality, we have the 
following definition: 

Àn n-dimenstonal Euclidean space E, is a set of objects called vectors 
which satisfy properties described above under subheads a to c, with the 
property that there exist n linearly independent vectors while every set of 
n 4- 1 vectors is linearly dependent. 


In E, we have U — (uus, . . . , u;) and 
ey us, « vs yg) (auos, s «s 0m) 
(ua, «.. Ma) P (Ua,Uo, «s Uu) 9 (ua v. Wo F9; os, Vs P vu) 
(U,U2, - Un) * (UyUs, «cs g0n) 55 aug e ats co buts 


À basis for E, 1s a set of n linearly independent vectors. Any vector 
in E, can be expressed uniquely as a linear combination of the vectors of a 
given bass. The set of n unit vectors (L0, . . .,0), (QUI, . . . ,0), 

Ce 00i vc )as albas Tor Bs. 

One linear condition in E; (such as aqui 4d- a2u» — b, where a, a», and 
b are constants) defines a line; one linear condition in E; defines a plane; 
the corresponding object defined by one linear condition in E, is called a 
hyperplane. A hyperplane H(A,5) in E, is the set of all vectors U such 
that À - U — b for a given vector À z£ O and a real number b (Fig. 2.5). 

A hyperplane divides E, into two half spaces which we denote by 


H*(Ajb) 2 (U|A-U — 6) 

H-(A5) —(U|A-U Sb) 
H*(A,b) is the half space, i.e., that portion of E,, that contains the vectors 
U for which A- U » b. H-(A,b) is the half space that contains the vec- 


tors U for which A- U «& b. 
For A ^ (2,—1) and b - 1, we have the inner product 


A*U —2u;— wu» 


FIGURE 2.5 
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with the two half spaces 


H*(A,j) 2-2u1—u;21 
and 

H-(A,b) -— 2u1 — M5 e JI 
as depicted in Fig. 2.5. "The points on the line 2u; — u» — 1 belong to 
both half spaces. 


3. CONVEX SETS 


A convex combination of the points U;, Us, . . . , U, is a point 
U e aUi: T aU» -F P Ade: T aUa 


where the o; are scalars, o; 2 0, and Y o; — l. A subset C of E, 1s con- 


1 
vex if and only if, for all pairs of points U, and Us in C, any convex 
combination 


U — o4U; c o3U» 


is also in C. 
Examples of convex sets are the whole space E,, a circle, and a cube. 
The set of points that form the boundary of a circle is not a convex set. 
We next show that, for a given convex set C, any convex combination 
of any number of points in C is also in C. Here we are given that the 
convex combination of any two points U; and U; in C is also in C. First 
we prove that 


U- o4U; Jr a9Us "E as3Us 
for o; 2 0, »» o; — l,isalsoin C. Let 
a - ES 
s o, -F a» 


for? — 1,2. We note that D oa; —- lando; » 0. We have 


U — oU; -- o5U ; cF o5Us — (o1 9 o3) (o5U1 4- o5Us) H- o5Us 
By hypothesis oU; -- o;Us is in C. Let 

U, — o,U; 4- o5U; 
Then 

U — (oi 4- a3)U; -- o35U; 


where U is a convex combination of two points in C, and hence U is in C. 
This procedure can be generalized for any i. 
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Theorem 1. Any point on the line segment joining two points in E, can 
be expressed as a convex combination of the two points. 


Proof. Denote the two points by U and V and let W lie on the line seg- 
ment joining U and V. "This line segment is parallel to the line 
defined by veetor U — V (Fig. 2.6). By the rules of vector addition 
we have, for some 0 € ^ € lI, 


Vos AU ew 


or 


(1—XV-4-AU-W 


which is the expression of the point W as à convex combination of V 


FIGURE 2.6 


Theorem 2 (converse of theorem 1). Any point thal can be erxpressed as 
a convex combination of two points in E, lies on the line segment jo?ning 
the two points. 


Proof. Here we are given 


Wo xg -—XY-3U 


or 
W-V 


A(U — V) 


whereO0 € A € 1. Hence the vector W — V is some positive mul- 
tiple of vector U — V, and these vectors cannot have the configuration 
as pietured in Fig. 2.7. Vector W — V must coincide in direction 


FIGURE 2.7 
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with U — V. Since the line segment joining U and V and the line 
segment joining W and V are parallel to the lines defined by U — b 
and W — V, respectively, point W must lie on the line segment joining 
U and V. 


By Theorems 1 and 2 we see that geometrically a convex set is one 
which contains all line segments joining any two points in the set. The 
sets shown in Fig. 2.8a and b are convex sets, while those of Fig. 2.8c and 
d are not. 


(5) (c) (d) 


(a) 


FIGURE 2.8 


À point U in a convex set C is called an extreme point if U cannot be 
expressed as a convex combination of any other two distinct points in C. 
Every point on the boundary of a circle is an extreme point of the convex 
set which includes the boundary and the interior of the circle. If the 
convex set did not include the boundary, then this set would contain no 
extreme points. "The extreme points of a triangle are its vertices. 

The convex hull C(S) of any given set of points S is the set of all 
convex combinations of sets of points from S. C(S) is the smallest convex 
setcontainng S. If Sis just the eight vertices of a cube, then C(S) is the 
whole cube, or, if S is the boundary of a circle, then C(S) is the whole 
circle. 

If the set S consists of a finite number of points, the convex hull of S 
is termed a convex polyhedron. |C(S) of the eight vertices of a cube is a 
convex polyhedron. If Cisaclosed and bounded set with a finite number 
of extreme points (e.g., a convex polyhedron), then any point in the set 
can be expressed as a convex combination of the extreme points. Thus C 
is the convex hull of its extreme points. 

A set of vectors S is called a cone if, for every vector U in S, AU is in S, 
where À is a nonnegative number. Examples of a cone are the whole 
space, the origin, and the set S of Fig. 2.9. "We note that a cone contains 
the origin, since ^ can equal zero. 

A convex cone is a cone which is convex. "The cone of Fig. 2.9 is not 
convex. "That part of S in the first quadrant is a convex cone. 

A simplex is an n-dimensional convex polyhedron having exactly n -- 1 
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vertices. "The boundary of the simplex contains simplices of lower dimen- 
sion whieh are called simplicial faces. "The number of such faces of 
h-P 
$1 
dimension it 1s a line; in two dimensions it is a triangle; and in three 
dimensions it is a tetrahedron. "The equation of a simplex with unit 
intercepts is 


quat) 


dimension 4 is ( ) A simplex in zero dimension is a point; in one 


For m — 3 we have the tetrahedron with vertices (0,0,0), (1,0,0), (0,1,0), 
and (0,0,1). 


FIGURE 2.9 


4. LINEAR INEQUALITIES 


Let us eonsider the following set of five linear inequalities in. two 
dimensions: 


(a) Ti ze) 
(b) q$2 0 
(c) Pipe (4.1) 
(d) £$q1— 22-1 
(e) —zi d 22 € 0 


We wish to determine which points (1,275) in two-dimensional Euclidean 
space satisfy the inequalities (4.1). We first note (Fig. 2.10) that the 
first two inequalities limit our search to the positive quadrant; i.e., 


1 The notation dem represents the number n!/m!(n — m)!. 
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only those points having nonnegative coordinates need be considered. 
Inequality (4.1a) is satisfied only by points lying on or to the right of the 
line z; 7 0; inequality (4.15) is satisfied only by points lying on or above 
the line z; — 0. The only points which simultaneously satisfy both 
inequalities are those lying in the first, or positive, quadrant. Each 
inequality is satisfied by all the points lying in one of the half spaces 
that is generated when we treat the inequality as an equality. "We note 


FIGURE 2.10 


the solution half space by drawing arrows from the dividing hyperplane 
into the solution space. We next treat inequality (4.lc) as an equality 
and draw the line corresponding to z; 4- x — 1. We determine that the 
half space which contains all the solutions to inequality (4.1c) lies upward 
to the right, by noting that the origin (0,0) does not satisfy the inequality ; 
i.e., the solution space does not contain the origin. "The solution space 
to inequalities (4.1a) to (4.1c) is that part of the positive quadrant whose 
points simultaneously satisfy the three inequalities. For inequality 
(4.14), we draw the line z; — z» — 1 and note that the half space whose 
points satisfy the inequality does not contain the origin. "The line which 
corresponds to inequality (4.1e), —z; -- 2x; — 0, contains the origin. 
To determine which of the associated half spaces represents the solution 
space for inequality (4.1e), we take any point, say (1,0), and note whether 
or not it satisfies (4.1e). Since (1,0) does satisfy (4.1e), the half space 
which contains (1,0) is the solution space for (4.1e). Only those points in 
region C described by the hatched lines simultaneously satisfy the five 
inequalities. "The solution space C is an unbounded, polygonal convex 
region whose boundaries are portions of the lines which correspond to 
inequalities (4.15), (4.1d), and (4.1e). Inequalities (4.1a) and (4.1c) do 
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not enter into the determination of the solution space. "They are 
redundant to the system (4.1), but only some type of analysis, such as the 
one above, will reveal this fact. 

We have interpreted each of the inequalities of (4.1) as defining a 
hyperplane in two-dimensional space. An alternate interpretation is 
obtained by writing the matrix representation of the system (4.1). We 
let matrix 


be the matrix of coefficients, and write the matrix inequality 


1 0 0 
D odes 0 
1-5 e zu (4.2) 
Is 1 
1o 0 


By multiplying the matrix À by the column vector whose elements are 
zi and zs, we have the five inequalities of the original system. . Here, 
inequality (4.1e) has been multiplied by —1 to reverse the direction of 
the inequality. We denote the first column of A by P; and the second 
eolumn by P»; that is, 


1 0 
0 1 
Pi -— i and P; -— il 
] —1 
1 —2 
and let 
0 
0 
Po,-—| 1 
I 
0 


the column vector of constants. "We can then write the system (4.2) as 
zjP, 4 z;P. P, (4.3) 


Here we interpret the column vectors P;, P», and P, as points in five- 
dimensional space, and the problem is to determine the weights z; and 
z» such that all the conditions defined by (4.3) are satisfied. 
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In general, the points which satisfy a linear inequality of the type 
QGii-F06i1t94d- ^c d aux. oi 


define a half spacein n-dimensionalspace. The joint solution space to the 
set of inequalities 


üyyby. -]- gsfg -F oc o*o* cR ana zd 
O»iXi -d- Ge»X» d- 5 coc P Gata 2n b» (4.4) 
GmX1 SE Qm 2X2 - ib SE OmnXn zt b, 


is a convex region in n-dimensionalspace. (We shall prove the convexity 
of the solution space in Chap. 3.) Since the direction of the inequality 
can be reversed by multiplication by —1, any set of inequalities can be 
written in the form of (4.4). The set of inequalities (4.4) can be trans- 
formed into a set of equalities by subtracting from each inequality an 
unknown nonnegative number. "These numbers are termed slack vari- 
ables. For (4.4), this transformation yields the set of equalities 


xot 9udasws cp oce spei mu — b, 
üpiti 4 dssty ost Ra: — Xn42 — bs 
Gm Ss Qm 2X2 uis A EC 4 Omni — W:ondm Us 
where z44;27 0(2 — 1, .. . , m). Since we can represent each variable 


t; as the difference of two nonnegative variables,! the system (4.4) has 
the following equivalent system in terms of nonnegative variables and 
equalities: 


Gu(z, — 2p) «- au(x; — 25) 4- c c d az, — 2.) — $44 -b 
an(r, — 31) c as(zx; — 33) 3: üt, — X4)  — Xaia zb. 
G»i(x, — 21) 4d- ass(x, — 23) - aas, — x.) — farm t Da 
where 

qy--,—u.. 

45 e) qesede n 

z 020 

a4; 2.0 DESSIR m 


! The reader should note that any number can be written as the difference of two 
nonnegative numbers. 
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Àn inequality of the form 
üxxricib pda 99 5 p iwi $03 


can be transformed into an equality by adding to it an unknown non- 
negative number (slack variable), e.g., 


ipt cie qoa t v Eau XZuq4l — b, 


where z,4,; 2 O. 
It should be noted that the solution space for a set of inequalities 
might be void. "There are no solutions to the set of inequalities 


Zip d 
2z1 4- 222 — 3 


As a mathematical problem, the general linear-programming problem 
can be described as follows: We are given a convex set which is defined 
by a set of linear constraints in n-dimensional Euclidean space. From 
all the points that belong to the convex set, we wish to determine a 
subset of points (which will contain either one or many points) 
for which a linear objective funetion is optimized. To illustrate how this 
can be done for two-dimensional constraints, we have represented in Fig. 
2.11 the convex set C, that is, the solution space, for Eqs. (4.1). Let us 
determine the subset of points for which the linear objective function 


FIGURE 2.11 
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2x, 4- 215is à minimum. In Fig. 2.11 we have also drawn the line 
2r; --F 2x3 — b 


for b — 12, 6, and 2. The intersections of these lines with C have been 
represented by heavy lines, with the last intersection being a point. 
Every point (ri,25) of the first intersection will yield a value of the objec- 
tive function equal to 12; all points of the second intersection yield a value 
of 6; and finally we see that the last intersection, which consists of only the 
point (1,0), gives the minimum value of 2. Hence, out of the infinite 
number of points contained in C only one point minimizes the objective 
funetion. "This point is also an extreme point of C, and the full conse- 
quencees of this will be discussed in Chap. 3. For any other objective 
function we can similarly determine the subset of points for which it is 
optimized. We need only to plot the objective function for some 
particular value and then move this line parallel to itself in the appropri- 
ate direction for minimizing or maximizing, as was done in Fig. 2.11. 
We move the line until the intersection of the line and convex set 1s reduced 
either to à point, a portion of the boundary of C, or until it is apparent 
that the maximum or minimum value is infinite. "These last two situa- 
tions are illustrated by minimizing 2z; — 2r» and maximizing 22, 4- 2x» 
for C of Fig. 2.11. If our convex set were a convex polyhedron, then all 
objective functions would have a finite minimum or maximum. 


5. SOLUTION OF A SET OF SIMULTANEOUS LINEAR EQUATIONS 


Any set of simultaneous linear equations has a convenient representation 
using matrix notation. The system 


01171 3-015349 1 75 basta R04 
Ost -F a3. *** d dat. 5; 
MICE MODE ud 
Gmi E Qm 2X2 T ded Eu Q5ndn — ba 


can be written as 


AX —b 
where 
Ti biN T 
A -— (aj) 2A Em : and Du 
Xn Is 


1 The reader is referred to the discussion of matrix multiplication in Sec. 1 for the 
validity of this representation. 
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If A is square (that is, if m — n) and nonsingular, the solution vector is 
given by 


X — A-!b 


There is à simple computational scheme that can be applied to any non- 
singular system to obtain the solution vector and/or the inverse. "This 
procedure, the complete elimination method of Jordan and Gauss, has a 
finite number of steps, or iterations. In just m(-m) iterations, the 
procedure multiples the system (5.1) by A-! to obtain X — A-!b. If itis 
so desired, A^! may be explicitly developed at the same time. We shall 
illustrate the method by an example. 
Consider the set of three equations in three variables: 


X1 4 qe —— pg m 2 
—2zi ES ND d- ba -- 3 (5.2) 
€1-F Xs x.- 6 


Let 
p b. 
A. se LT 1 
jit 1 


be the matrix of coefficients. It is easy to verify that À is nonsingular. 
We can rewrite (5.2) as the matrix equation 


Dod Lui 2 
——- lj|izs]2[29 
DI 1j Ns 6 


or, by letting 


1 1 —1 D 
Pi E —2 P; E 1 P; —— 1 P, I 5 
1 1 1 6 


we can rewrite (5.2) as 
z,;P, 4 rP.dFoxP; P, 


Since À is nonsingular, the set of vectors P,;, Ps, and P; is linearly inde- 
pendent and hence forms a basis in three-dimensional space.! "To solve 


! To prove that the vectors Pj, P», . . . , P, of a nonsingular matrix A form a 
linearly independent set, we first assume that the set is linearly dependent. Hence, 
for some set of o; we must have 


o1 Pi J- o3P5 4 QUE FR 4- ve,P. — [0) 


with at least one o; »£ 0. We can then solve for some vector P;, for example, P;, in 
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(5.2), we need to find the unique linear combination (1.e., the numbers 
4, 3», and z;) of P;, P», and Ps which equals vector Py. We shall first 
obtain the solution vector to (5.2). "The complete elimination procedure 
suecessively eliminates the first, the second, the third, ete., variable from 
all equations except the first, the second, the third, ete., respectively. 
For (5.2), the first step is to eliminate the first variable from all the equa- 
tions except the first. "This elimination is accomplished by adding and 
subtracting suitable multiples of the first equation to the second and 
third equations. 


Step 1 
q1-d- X. $472 

—22z4 d xs 43-233 — 3 
q1-F 4s-- X8 6 


Since matrix À is nonsingular, Eqs. (5.2) can be listed in such an order 
that the coefficient of the first variable in the first equation does not equal 
zero. Here a1 — 1l. To eliminate z; from the second equation, we 
multiply the first equation by 2 and add it to the second. To eliminate 
the first variable from the third equation, we multiply the first equation 
by —1and addittothe third. "These two substeps transform the original 
equations into the equations 


Xil — X. 92 
Odo e qaum Wf 
Dy cet 


Step 2. Here we eliminate the second variable from all the equations 
obtained as a result of Step 1 except the second. Again the equations 
can be arranged to make the coefficient of z» in the second equation not 
equal to zero (this would have been done if the third equation had origi- 
nally been the second). Since at the start of an iteration it ls convenient 
to make the coefficient of the variable being eliminated equal to 1, we 


terms of the other vectors to obtain 


Pi em BP» *- ILS m SE [69 925 
where 8; — —oj/oy. We substitute this expression for vector P, in matrix A. Then 
by successively adding to the first column —8,P;forj — 2,3, . . . , n, we reduce the 


first column to the zero vector. From the properties of determinants given in Sec. 1, 
the addition of these vectors does not change the value of the determinant, and since 
a column is now equal to zero, the value of the determinant is zero. However, this 
contradicts the given information that A is a nonsingular matrix. Hence, our assump- 


tion of linear dependence has led to a contradiction, and therefore the set of vectors of 
A must be linearly independent. 


Sec. 5] MATHEMATICAL BACKGROUND 4l 


divide the second equation by 3. "The coefficient 3 is referred to as the 
pivot element. (In Step 1 the pivot element wasa;; — 1. | We then have 
to multiply this transformed second equation by —1 and add it to the 
first to obtain the equations 


2m a 
2 Mn - 
223 —4 
Step 3. Herethe pivot elementis2. We first divide the third equa- 
tion obtained in Step 2 by 2, and then multiply it by 24 and L4 and add 
the corresponding results to the first and second equations, respectively. 
'The system (5.2) has been reduced to the solution 
21 ze 
X2 -—3 (iu) 
UOS 2 
'The operations of the complete elimination procedure have transformed 
the matrix of coefficients 


]- 71d 
Avwesgeq 4 
E 3g 


into the identity matrix [the matrix of coefficients for the transformed 
Eqs. (5.3)] 


Ix 0070 
I 0 ll 0 
UC UN 


We then see that the total transformation accomplished by the complete 
elimination procedure is equivalent to multiplying the system (5.1) by 
f ce 

It is very often necessary not only to solve a system of simultaneous 
linear equations, but also to obtain the inverse of the matrix of coefficients. 
This is accomplished by attaching an m X m identity matrix to the right 
of the original matrix of coefficients and applying the elimination trans- 
formations to the extended matrix. "The inverse 1s generated in place of 
the identity matrix. lf we write the partitioned matrix 


(A|I| P) 


(i.e., the matrix À with an m X m identity matrix written beside it and 
the P, vector attached to the end) and apply the complete elimination 
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transformation, we have 
(A-1AÀ | A-IT| AP) — (I| A^! |X) 


We shall illustrate this procedure for the previous example. "The par- 
titioned matrix is 


TESTE E00 2 
—2 1 1 0.1 0 3 
| TE: 1 0 9.1 6 


The successive steps, as before, yield 


Step 1 

1 1 —1 1 70.20 2 

0 3 -1 2 l1 0 ví 

0 0 2 —1 0 1 4 
Step 2 

l 0. —2$ L$ 14 0 3 
01 -k| M o0, 
0 0 2 —1 0 1 4 
Step 3 

4:000 0 —l$ l4 I 

n 1e dg M 3 
Oosnpéd- ees cesse tL NUM 


We then have 


MEL 
A3-| HX M 
«419-00 1$ 
The unique linear combination of the vectors P,, P;, and P4 which 
equals P, has been determined to be 


1P; 4-3P,-.-2P; —- P, 


Since the vectors P,, P», P; form a basis in three-dimensional space, any 
other vector in three-space can also be expressed as a unique linear com- 
bination of these three vectors. For any such vector, say P4, we wish to 
determine y;, y», and y; such that 


yiPi sis J2P» - ysP3 —P, 
Or 
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To solve for Y, we must multiply (5.4) by A-! to obtain 


A^CIAY — A-CP, 
M es A-!P, 
For example, let 
E 
P, e —2 
4 
Y is given by 
Ü. Ri 8 4 2 
Y-( x — X xp -2]-2l2? 
-M 0 M/N 4j W 


P, expressed as a linear combination of P;, P;, and P; is then 
2P, 4- 2P, -- 0P4 E P, 


Here P, is equal to à linear combination of only two of the three basis 
vectors. "The three-dimensional vectors P;, P;, P, are linearly depend- 
ent since 


2P,--2P,—P,-0 


with the coefficients of vectors P;, P», and P, not all zero. "The situation 
where a vector can be expressed as a linear combination of less than m 
vectors from a given basis is termed degenerate. 

A system of linear equations ÀÁX — b is said to be Aomogeneous i 
b — OO. Such a system always has the trivial solution X — O. 


REMARKS 


Much of the material in Sec. 1 1s from Hildebrand [55a], and much of that in Sec. 2 
is from Kuhn [67]. For additional reading concerning the theory of convex sets, the 
reader is referred to Part Three of Koopmans [65], Kuhn and Tucker [68], Hadley [526], 
and Nef [77d]. 


EXERCISES 


1. Given the matrices 
2 6 0 —4 
ar e 3 H Ps ( 5 


2 
c-[-1 
9 


compute the following: 


(A -- B), CD, DC, |C|, |D|, C^'*, D^, C', D', (CD)', D'C', (CD)! 


C1 05 m 
t2 com 
DLE d 
Jg 

ll 
EE 

| 
t2 Oo r2 
Oct Ot 
T t0 
Dd 
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2. Graph the following vectors: 


im M e) 
U--V,U-V,2U— 14V U— V-W,U-- V -W 

3. Graph the convex hull of the points 

(0,0), (1,2), (—1, —D, (—2,2), (54), (0,3), (—1,1), 0$,, (—1,2), (2,5) 


Find convex combinations of the extreme points which express those given points that 
are interior points of the convex hull. 

4. Graph the following linear constraints, mark the area that satisfies the con- 
straints, and determine the extreme points of this convex set: 


—2z; 4-5z; —10 € 0 
2x; d4- z;,— 6 €0 
zji4d-22;,— 220 
—zji-d-3z,— 3&0 


5. Write the system of inequalities of Exercise 4 as a system of equalities in non- 
negative variables. 
6. Assume a given basis of the vectors 


1 1 2 
p,-[0 P,-2[1 Ps -—[T 
1 1 1 
Compute the inverse of the matrix associated with this basis and determine the linear 
jl 
combination of the basis vectors that equals vector P, — [ 3 
4 


T. (The convex region shown in the figure below is the set of solutions determined 
by a set of linear inequalities. At what point or points are the following linear func- 
tions optimized? 

a. 0$; 4- zs — 1, to be a maximum 

b.  3z, — zs -- 6, to be a minimum 

C. —2z, — 2x5 4- 2, to be a maximum 


2 


FIGURE E7 
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8. Given the set of equations 


QiiXi d disX2 d- Qasra — 


] 
& 
z 


Q»iX| -- Go»X5 -d- assa — 


] 
& 
z 


GaXi -d- GaoX» -F Gast — Gao 


Assume that the given matrix of coefficients (a;;) is nonsingular. Reduce the system 
to the unknown vector (zi,2s,23) by the complete elimination procedure, and develop 
general formulas for the transformations that are performed in each iteration. 

9. Solve the following set of equations, using the complete elimination formulas 
developed in Exercise 8. Also compute A^! by means of the adjoint matrix and check 
the result by computing AA-^!. 


2x, 4 2r, — da 4 
3a; — zs. — 903 — 7 


zi -d- 224-223 — 8 


10. Solve the following linear-programming problems by graphical methods: 
a. Constraints: 


2z; — Qo —2 


zji d- 22; € 8 
Zi 20 
252210 


Objective functions: maximize rz», maximize 3z; -- 2z», minimize 2z, — 2z5, 
maximize 2x, 4- 4x», minimize —3z; — 2z». 
b. Constraints: 


9z, 4 227; € 6 
qy-— $$» —l 
—z|— 27; 2l 
Zi 22230) 
25 20 


Objective function: maximize 2z; — 625. 
c. Constraints: 


z;— 93 € 6 
2z; 4-42; 2 8 
z; — 39z5 2 —6 
Zi zx) 
2220 


Objective functions: maximize 2z; 4- 32», minimize z; 4- 2z», maximize z; — 2r», 
maximize z, — 3r», maximize z; — 6z;. 


46 INTRODUCTION [Chap. 2 


d. Constraints: 


Sa siot aa iM NS 


zji-c-r;— rl 
Eo 20 
T9 20 
23-240 


Objective functions: maximize z; J- z» d- zs, minimize z; 4 z;,, maximize z; 4 
o 323. 
11. For a given set of inequalities 


Oir cis se dente SS L DEEP UT 
go cU" ed ern 


discuss conditions that indicate that an inequality does not form part of the boundary 
of the solution space. 

12. Define the intersection of two sets K; and K; to be the set of points belonging 
to both K; and K». If K; and K:» are both convex, prove that their intersection is also 
convex. 

13. For a convex cone, show that the positive sum of any two vectors in the cone 
is also in the cone. 

14. Solve Exercise 5 of Chap. 1. 


PART TWO / METHODS: 
THEORETICAL AND 
COMPUTATIONAL 


CHAPTER 3 / THE GENERAL 
LINEAR-PROGRAMMING PROBLEM 


1. THE LINEAR-PROGRAMMING PROBLEM 


The general linear-programming problem is to find a veetor (xi25, . . . , 
Xj ... ,25) Which minimizes the linear form (i.e., the objective function) 

Coro Ooutsse scoroepog spe eme, (IL) 
subject to the linear constraints 

d; VM uh p MET (1.2) 
and 

Oscricidesscs- s saspe qup cscbsüi dass bi 

19 4- da»3s Roc 7 7 -bouX d coc 0 EROü:X — bs 

SU OW Ra De - DRUG IC AR dC D EDNMARTDT OS SNL IPSIS (DIT SPI Y des 

QüX| -F Gi. d *ocb oam; d T Gus - bi 

Qmv1 4 Qm 2X2 n ^ se 4 Gm; SE dc m ÜmnXga — lr 


where the a;;, b;, and c; are given constants and m « n. We shall always 
assume that the Eqs. (1.3) have been multiplied by —1 where necessary 
to make all b; 2 0. Because of the variety of notations in common use, 
one will find the general linear-programming problem stated in many 
forms. "The more common are the following: 


a. Minimize 
n 
» Cjt; 
1 


subject to 
m0 qx 25 x s 
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and 


n 
» Qi; — b; i — i 2, pra e m 
4-1 


b. Minimize 


cX 
subject to 
X0 
and 
AX — b 
where c — (61,65 .. . ,€4) 1s 8 row vector, X — (x45... 22) 18.8 
column vector, À — (aij), b — (51,05, . . . ,0,) is a column vector, and 


O is an n-dimensional null column vector. 
c. Minimize 


cX 
subject to 


X20 
and 
xjPQ do XP. 5c: oc ZAR. - Ps 


where P; for; — 1, 2, .. . , n is the jth column of the matrix À 
and P, zb» 


2. PROPERTIES OF A SOLUTION TO THE LINEAR-PROGRAMMING 
PROBLEM 


In this section we shall state a number of standard definitions and describe 
the more important characteristics of a solution to the general linear- 
programming problem. Much of this material and that in Chap. 4 is 
contained in Dantzig [17] and Charnes, Cooper, and Henderson [12]. 


Definition 1. ^A feasible solution to the linear-programming problem is 
& vector X — (rjrs, . . . ,z,) which satisfies conditions (1.2) and 
(à Psy 


Definition 2a. A bas?c solution to (1.3) is a solution obtained by setting 
n — m variables equal to zero and solving for the remaining m varia- 
bles, provided that the determinant of the coefficients of these m 
variables is nonzero. "The m variables are called basic variables. 


Definition 2b. — A basic feasible solution is à basic solution which also satis- 
fies (1.2); that is, all basie variables are nonnegative. 
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Definition 3. .À nondegenerate basic feasible solution is a basie feasible 
solution with exactly m positive z;; that is, all basic variables are 
positive. 


Definition 4. A minimum feasible solution is à feasible solution which 
also minimizes (1.1). 


Unless otherwise stated, when we refer to a solution, we shall mean 
any feasible solution. 


Definition 5. A near functional f(X) is a real-valued function defined 
on an n-dimensional vector space such that, for every vector 
X — aU -- 68V, f(X) ^ f(aU -- 8V) — af(U) 4- 8f(V) for all n-dimen- 
sional vectors U and V and all scalars a and 8. For example, let 
U-(93),V—(660)020-—14,8 — —25, and f(X) 22x, -- zx. We 
have oU 4- 68V — (—1,—3) and f(aU -- 8V) — —5; af(U) 2 7 and 
pem Ton 


We note that the objective function (1.1) is a linear functional for 
those X satisfying (1.2) and (1.3). 


Theorem 1. Te set of all feasible solutions to the linear-programming prob- 
lem is a convez set. 


Proof. We need to show that every convex combination of any two 
feasible solutions 1s also a feasible solution. (The theorem is true, of 
course, if the set of solutions has only one element.) Assume there 
are at least two solutions X; and X;. We have 


AX; -— b for X; m Oo 
and 
AX, —Xb for X; p O 


For0 €oa € I, let X — oX; 4- (1 — o)X;beany convex combination 
of X; and X;. We note that all the elements of the vector X are non- 
negative; that is, X 2 O. We then see that X is a feasible solution, 
for we have 


AX — A[oX, 4- (1 — o)Xs] ^ aAX, 4 (1 — o)AXs 
—oab-d-b—ob-b 
In a similar manner, one can prove that the sets of solutions to 


the inequalities (4.4) and the equalities (5.1) of Chap. 2 are convex 
sets. 


We shall denote the convex set of solutions to the linear-programming 
problem by K. Since K is determined by the intersection of the finite 
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set of linear constraints (1.2) and (1.3), the boundary of K (if K is not 
void) will consist of sections of some of the corresponding hyperplanes. 
K will be a region of E, and can either be void, a convex polyhedron, or a 
convex region which is unbounded in some direction. If K is void, then 
our problem does not have any solutions; if itis a convex polyhedron, then 
our problem has a solution with a finite minimum value for the objective 
function; and if K is unbounded, the problem has a solution, but the 
minimum might be unbounded. — Valid linear-programming models should 
yield K's of the second or possibly the third type, as they are models of 
situations that have a number of possible solutions. By Theorem 1, if a 
problem has more than one solution, it has, in reality, an infinite number 
of solutions. Out of all these solutions, it 1s our task to determine the 
one which minimizes the corresponding objective function. "This work is 
somewhat simplified by the results of Theorem 2 below. Before pro- 
ceeding with this theorem, we should note the following: If K is a convex 
polyhedron, then K is the convex hull of the extreme points of K. That is, 
every feasible solution in K can be represented as a convex combination of 
the extreme feasible solutions in K. (By definition, a convex polyhedron 
has a finite number of extreme points.) An unbounded K also has a 
finite number of extreme points, but not all points of K can be repre- 
sented as convex combinations of these extreme points. For ease in dis- 
eussion, we can assume that all our problems have a K that 1s a convex 
polyhedron. As will be shown in later chapters, computational devices 
exist that determine whether K is void or whether a problem has an 
unbounded minimum. (For computational purposes, we always assume 
that K is a convex polyhedron.) 

With the assumption that K 1s a convex polyhedron, we can surmise 
from the above discussion that we need only to look at the extreme points 
of the convex polyhedron in order to determine the minimum feasible 
solution. We prove this with the following theorem: 


Theorem 2.  T'he objective function (1.1) assumes its minimum al an extreme 
point of the convex set K generated by the set of feasible solutions to the 
linear-programming problem. | If it assumes its minimum at more than 
one extreme point, then it takes on the same value for every convex 
combination of those particular points. 


Proof. Since we have assumed K to be a convex polyhedron, K has a 
finite number of extreme points. In two dimensions, K might look 
like Fig. 3.1. Let us denote the objective function by f(X), the 
extreme points by X,, X;, . . . , X,, and the minimum feasible 
solution by X. This means that f(X)) X f(X) for all X in K. If 


X, is an extreme point, the first part of the theorem is true. Suppose 
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X, is not an extreme point (as indicated in Fig. 3.1). We can then 
write X, as a convex combination of the extreme points of K, that is, 


for o; 2 0 and » o; — l. "Then, since f(X) 1s a linear functional, we 
i 
have 


JO RM oX;) — f(aiXi -- osXs Fc o ou X;) 
s ouf (Xi) T esf (X5) eue ere af (X5) Um 
(2.1) 


where m is the minimum of f(X) for all X in K. 


FIGURE 3.1 


Since all à; 2 0, we do not increase the sum (2.1) if we substitute 
foreach f(X;) theminimum of the valuesf(X;). Let/f(X,) — min f(X.). 


Substituting in (2.1) we have, since Y o; — 1, 


FUEo Lx conf) cce FOL) cspenirgf Sacs JO) 
Since we assumed f(X,) € f(X) for all X in K, we must have 


f(X) — f(K,) — m 


Therefore, there is an extreme point, X,, at which the objective func- 
tion assumes its minimum value. 

To prove the second part of the theorem, let /(X) assume its 
minimum at more than one extreme point, say at Xi, X», . . . , X,. 
Here we have f(Xi1) — f(X5.) 2 - - - 2 f(Xj) — m. If X is any 
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convex combination of the above X;, say 


for a; 2 0 and Y o; — 1, then 


J(X) m J(oxXs zm € Apu soe tuD a X,) 
- ef) adf) Eoo ob af) 0 Y am 
- 


The proof is now completed. By making the obvious changes, the 
theorem can be proved for the case where (1.1) is to be maximized. 
By Theorem 2, we need only to consider the extreme points of K in 
our search for a minimum feasible solution to the linear-programming 
problem. 


Recall that a feasible solution is a vector X — (ri2s, . . . ,2,), with 
all x; 2 0, such that 


q$4P1 -Ez;P.-Fo55o0oB XP - Ps 


Assume we have found a set of k vectors that is linearly independent 
and that there exists a nonnegative combination of these vectors that is 
equal to Ps. Let this set of vectors beP;, P5, . . . , Pa. Wethen have 
the following theorem: 


Theorem 3.  /f a set of k C m vectors Pj, Pj, . . . , Pi can be found that 
is linearly independent and. such that 
ziPi deWwuP, pcc oÓbE-P.- P, 
and all 2; — 0, hen the pow X — (ys «.— 0, 0) 28 0n 
extreme point of the convex set of feasible solutions. Here X is an n- 
dimensional vector whose last n — k elements are zero. 


Proof. Suppose X is not an extreme point. "Then, since X is a feasible 
solution, it can be written as a convex combination of two other points 
X, and X, in K. We have X — oX, -- (1 — o)X; for 0 « a « 1. 
Since all the elements z; of X are nonnegative and since 0 « a « 1, 
the last n — k elements of X; and X; must also equal zero; that is, 


X4 -— (mp. dre 4 RA UR Mee wa ,0) 


X; — (zin, 2. un 0,0, .. . 0) 
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Since X, and X; are feasible solutions, we have 


AX; —b 


AX,.—b 


Rewrnting these equations in vector notation, we have 


TOPPED E 
and 

qUPdqupie5s4mp P, 
But Pj, P5, . . . , P; are linearly independent, and hence P, can be 
expressed asa unzque linear combinationintermsof P, P» . . . , P.f 


Thisimpliesthatz; — rz, — z;?. "Therefore, X cannot be expressed 
as à convex combination of two distinct points in K and must be an 
extreme point of K. 


Theorem 4. /f X — (xjyzs, . . . ,2,) is an extreme point of K, then the 
vectors associated with positive x; form a linearly independent set... From 
this it follows that, at most, m of the x; are positive. 


Proof. Let the nonzero coefficients be the first À coefficients, so that 


k 
» zx;P; —» P. We prove the main part of the theorem by contra- 
i1 
diction. Assume that P; P» ..., P, are linearly dependent. 


'Then there exists a linear combination of these vectors which equals 
the zero vector, 


diPi E d3P; - d ugeko 4- d, P, -- O (2.2) 
with at least one d; z£ 0. From the hypothesis of the theorem, we 
have 

Ai)PydoaxBico*cocbBXPRReDP, (2.3) 
t This can be shown as follows: Let Pj, P», . . . , Py be a set of linearly inde- 


pendent vectors and assume that we can represent the vector P, in terms of these 
vectors by two different linear combinations, for example, 


€jP, 43- eP2 4 5 -cFexuPR — Po 
and 
fJiPicbfPs.-doe s cfÁPR-Po 


To show that these combinations have to be identical, we subtract the second from 
the first to obtain 


(e — f)Pi *E (ea — f2)P» WP O5 Boe (6x — fx)Pk —0, 


By the definition of linear independence we must have each e; — f; — 0, which implies 
tthetues s 
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For some d 7 0, we multiply (2.2) by d and add and subtraet the 
result from (2.3) to obtain the two equations 
k k 
Y x; P; ns d 2s d;P; m P, 
i-l i-c1 
k k 
pr zP; —d b d;Pi -— P, 
i-cl i-1 
We then have the two solutions to (1.3) (note that they might not be 
feasible solutions): 


Xo cdd n d otov edd. 00S v9) 
and 
X» -— (zi — dd, H PD dd», UEOEES EDS dd;, 0, Cata. ,0) 


Since all z; 7» 0, wecan letd beas small as necessary, but still positive, 
to make the first k components of both X, and X; positive. Then X, 
and X; are feasible solutions. But X — 1$X, 4- L6X;, which con- 
tradiets the hypothesis that X 1s an extreme point. "The assumption 


of linear dependence for the vectors P;, Pj, . . . , P; has thus led to 
& contradietion and hence must be false; i.e., the set of vectors 
P,P,...,P;islinearly independent. 


Since every set of m -- 1l vectors in m-dimenslional space 1s 
necessarily linearly dependent, we cannot have more than m positive 
zi. Forassumethat wedid. "Then the above proof of the main part 
of the theorem would imply that there exist vectors Pi, . . En 
P,4,; that are linearly independent. 

We can, without any loss of generality, assume that the set of 
vectors Pj, P, . . . , P, of the linear-programming problem always 
contains a set of m linearly independent vectors. If this property is 
not evident when a particular problem is being solved, the original 
set of vectorsis augmented by a set of m linearly independent vectors, 
and we then seek a solution to the extended problem. This procedure 
will be explained in detail in the succeeding chapters. 


S, 


Corollary 1. Associated with every extreme point of K is a set of m linearly 
independent vectors from the given set Pj, Py, . . . , BP, 


Proof. 'Theorem 4 has shown that there are k X m such vectors. For 
k — m,thecorollary is proved. Assumethat k « m and that we can 
find only additional veetors P4,;,, . . . , P, such that the set 


Pos Dp T 


for r « m is linearly independent. "Then this implies that the 
remaining » — r vectors are dependent on Pj, . . . , Bj. But this 
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contradiets the assumption that we always have a set of m linearly 


independent vectors in the given set of Pj, . . . , P,. "Therefore, 
there must be m linearly independent vectors Pi, . . . , P,, associ- 
ated with every extreme point, such that 

k m 

» z;P; 4 » OP; -— P, 

i-i i-E41 


We can sum up the preceding theorems by the following: 


Theorem 5. X — (xjzs . . . ,1,) is an extreme point of K 2f and only 
f the positive x; are coefficients of linearly independent vectors Pj in 
n 
b z;P; -— Po 
j-1i 


As a result of the assumptions and theorems of this section, we have: 


1. There is an extreme point of K at which the objective function takes 
on its minimum. 
. Every basic feasible solution corresponds to an extreme point of K. 
3. Every extreme point of K has m linearly independent vectors of the 
given set of n associated with it. 


L2 


From the above we can conclude that we need only investigate 
extreme-point solutions and hence only those feasible solutions generated 


by m linearly independent vectors. Since there are at most [m sets of 


m lhnnearly independent vectors from the given set of n, the value e 


is the upper bound to the number of possible solutions to the problem.! 
For large » and m it would be an impossible task to evaluate all the pos- 
sible solutions and select one that minimizes the objective function. 
What is required is à computational scheme that selects, in an orderly 
fashion, a small subset of the possible solutions which converges to a 
minimum solution. "The simplex procedure, devised by G. B. Dantzig, 
is such à scheme.? This procedure finds an extreme point and deter- 
mines whether it is the minimum. If it 1s not, the procedure finds à 
neighboring extreme point? whose corresponding value of the objective 


! See Saaty [88] and Quandt and Kuhn [85c] for discussions of the upper bound to 
the number of possible solutions. 
? The name simplex method is due to the use of the equation D. qz —d astaccons 


Jj 
straint in a geometric interpretation of this procedure, as described in Dantzig [17]. 
3 Two extreme points are said to be neighbors if they are joined by a boundary of 
the convex polyhedron. 
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function is less than or equal to the preceding value. In a finite number 
of such steps (usually between m and 2m), a minimum feasible solution is 
found. "The simplex method makes it possible to discover whether the 
problem has no finite minimum solutions or no feasible solutions. Itisa 
powerful scheme for solving any linear-programming problem. 

Before going into the validity and full computational aspects of the 
simplex procedure, we wish to introduce the following computational 
element of the procedure. 


3. GENERATING EXTREME-POINT SOLUTIONS 


Here we assume that an extreme-point solution in terms of m vectors Pj 
of the original set of n vectors is known. We can let this set of m linearly 
independent vectors be the first m, and let 


Ncecor mm c tont e ED 
be the solution vector. We then have 
XjP.-borPa 0 4 mp. Bo (8.1) 


where all z; 2 0. With this information, the problem is to determine, in 
a computationally efficient manner, a new extreme-point solution. (We 
shall, of course, assume that a different extreme solution exists.) Since 
the vectors P;, P», . . . , P, are linearly independent, they form a basis 
in m-dimensional vector space. We can then express every vector of the 
given n as à linear combination of these basis vectors. We can write 
Tri 
TgP; — P; Lx M NE 


1-1 
Assume that some vector not in the given basis, say P, has at least one 
element z; 5,41 » 0 in the expression 


X1, xb m X, m41P2 "n NE UE m Xu, malbe m B (3.2) 


Let 0 be any number, and multiply (3.2) by 8 and subtract the result 
from (3.1) to obtain 


(v o 021, 41) P1 RE (2 E 0x5, 4.41) P spe ote 
EDS (35. x poe i41) B -- QE e P, m) 
Thé yeotór X' 9 (2. 021 au dic POT D RIS Bra ci ads 


à solution to the problem, and if all the elements of X' are nonnegative, 
X' is a feasible solution. Since we want X' to be a feasible solution dif- 
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ferent from X, we restrict 0 to be greater than zero.! With this restric- 
tion, all the elements of X' that have a negative or zero z;, 4,41 will also 
have a nonnegative z; — 6r; 4,1. We need only concern ourselves with 
those elements having a positive zr; 44:1. We wish to find a 0 » 0 such 
that 


HE Cm D m4-1 m 0 (3.4) 
for all 2; «41 2-0. 
From (3.4) we have 


ONE 


Xi, m--1l 


and hence any 6 for which 


i Zi 
Q0 «0 « min : 
$0 T4 m4 


will give a feasible solution for (3.3). However, as we are looking for an 
extreme-point solution, we know by our theorems in Sec. 2 that we can- 
not have all the * 4- 1 elements of X' positive. We then must force at 
least one of the elements of X' to be exactly equal to zero. We see that, 
if we let 


(2 ES velut) 
io X4 m4l 


for zi, 441 » 0, then the element in. X' for which this minimum is attained 
will reduce to zero. Let this element be the first, that is, 


X1 


, 475 
0; — min — 
io Xd m4l T1, m4i 


We have now obtained a new feasible solution 


/ / / / 
z3P» E z,P; 4 ed 4E ZIP 3 TociPep ex: P, 
where 
, sew. 92 
X; — Xi — Oi m4a Qum Esse TIU 
/ 
Tl -- 0o 


1 'The reader should note that this restriction causes us to assume (for illustrative 
purposes) that associated with an zi,»4: » 0 is an z; 7» 0. "This, in general, is not 
the case and will be true only if nondegeneracy is assumed for all basic feasible solu- 
tions. As discussed in Chap. 4, a value of 6 — 01s an acceptable one; ie., some z; — 0 
for an zi; 54411 » 0. With 6 — O0, the transformation retains the old extreme point but 
selects a new feasible basis. In sum, the simplex procedure allows for all values of 
6 20. The reader should review the applicability of the above discussion to the 
case 0 — min (T;/2; m41) — O0 for $; 41 — 0. 

1 
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If all the z;, ,,; had been equal to or less than zero, then we would not 
have been able to select a positive 0 that would have eliminated at least 


one of the vectors Pj, . . . , P, from the basis. For this situation we 
obtain for any 0 7 0 a non-extreme-point feasible solution associated 
with them 4- 1 vectors P,, . . . , B5, P441. As will beshown in Chap. 4, 


this situation indicates that the problem does not have a finite minimum 
solution. 

ToshowthatX' — (xj .. . (25,2,,1) isan extreme point, we have to 
prove P5, . . . , P4, P441 are linearly independent. Assume they are 
linearly dependent. We can then find (from the definition of linear 
dependence) 


d3P» AE d;P3 - VM LE s ds Pu c Üncibuai em O (3.5) 
where not all d; 2 0. Since any subset of a set of linearly independent 
veetors is also a set of linearly independent vectors, P5, . . . , P,, are 
linearly independent. "This implies that d,41 z£ 0. From (3.5) we have 

€?Ps EE e5P3 m AIDE AE On P n m Pia (3.6) 
where 

ei — ut 

; ( MAPEN 


Subtracting (3.6) from (3.2), we obtain 


X1, mzybs d- (2, mds €3) P» 3E (3, mp —— €3)P3 pest 
x (£m, ml eR —-0 (3.7) 


Since P,, P» .. . , P, are linearly independent, all the coefficients in 
(3.7) must equal zero. But z;,4,41 was assumed to be positive. Hence 
the assumption of linear dependence for P», . . . , P,,,, has led to a con- 
tradietion, and these vectors must be linearly independent. 

In order to continue this process of obtaining new extreme feasible 
solutions, we need the representation of any vector not in the new basis 


P,jP;...,P,jinterms of this basis. From (3.2) we have 
1 
P, E LU (Pu xo. n»iiPs zu MERE eos wt bsu) (3.8) 
Let 
P, LX zijP 1 25;P» T PAL xs foBa (3.9) 


be any vector not in the new basis. Substitute the expression (3.8) for 
P, in (3.9) to obtain 


Pp; — c "s z Y, - Ps (ss — T3, 2 Poppy 


Ui m4 X1, m41 
Xi 


- [us a PIS 


Zi 
Vm, ^u) j EE 2 


Ti m4 X1, m1 
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The reader will note that the formulas for complete elimination 
required in Exercise 8, Chap. 2, are equivalent to the transformation that 
describes P, and Pjin terms of thenew basis. "The procedure for obtaining 
new extreme-point solutions is that of selecting a new variable to be intro- 
duced into the system, determining which variable has to be removed 
from the solution in order to preserve feasibility, and applying the com- 
plete elimination formulas to obtain the new solution and the new repre- 
sentations of the vectors not in the basis. "The criterion used to deter- 
mine which variable is to be introduced into the solution is a feature of 
the simplex procedure and will be considered in Chap. 4. 


Example. "We are given the following set of equations: 


n P. PIESPIT PL P CUPS 


92; — zs Fb 2xzs-d- ax 7 
2x, — Ac -cF zs ED 
—4z, — 3r» 4- 823 penc uti) 


We have as an initial extreme-point solution z; — 0, x — 0, z; — 0, 
X4 — 7, r$ — 12, ze — 10, which in vector notation is given by 


7P, 4- 12P; 4- 10P, — P, (3.10) 


Here the basis vectors P,, P;, P; are unit vectors. We wish to intro- 
duce vector P, to obtain another extreme-point solution. "The 
representation of P, in terms of the basis vectors is simply 


that 1s, 
341 — 3 X51 2 3g — —4 


If we multiply (3.11) by 8 and subtract the result from (3.10), 
we have 


(199 )P. E19 — 902)D, -F (10-46) P.-1-9P; 9. P, 332) 


Since z4; — 3 and z;5| — 2 are both positive, we determine 0, by 
evaluating, for these positive zii, 


Substituting this value in (3.12), we eliminate P, from the basis to 
obtain 


224 Pg -- 952 Ps 4 T8Pu e Ps 


1 We form the ratios for those ? in the current solution. Here ? — 4, 5, 6. 
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or the extreme-point solution z; — 74, x» — 0, 3 — 0,24 — 0,25 — ?24, 
2 Bd 
46 ÁM I 


If, instead of P;, we tried in a similar manner to obtain an extreme 
solution with P;, where 


—P, - 4P; 3 3P;s -— P; 


we would have developed the following expression for P, in terms of 
P,, Pp P P»: 


(7 -F 6)P, -- (12 4- 460)P, - (10 -- 36)P, 4- 6P; — Po 


From (3.13) we see that any 6 » 0 yields a feasible solution z, — 0, 
4. — 0, 424 — 0, x, — 7 4-0, 25 — 12 4- 40, z& — 10 4- 30. Here, 
since all zi» € 0, we do not obtain a new extreme-point solution. 

À more efficient way of interpreting the problem 1s as a trans- 
formation accomplished bv the elimination procedure. Here we 
detach the coefficients of the equations and set up the following 
tableau: 


(3.13) 


Pi P; P; P, iB; P, P, 0 
(3 —1 2 1 0 0 7 T4 — 6s 
2 —4 0 0 1 0 12 6 

—4 —3 8 0 0 1 


As we want to introduce P; into the basis, we again form the ratios 
ri/riforzi; » 0. Since 60, — 74 is the minimum of these ratios, we 
let the element 3 of P; be the pivot element of the elimination pro- 
cedure, as denoted by the circle. "That is, we shall eliminate z; from 
all the equations except the first. If we carry out the elimination 
transformation, we obtain a new tableau. Here z; — 74, x; — 224, 
Xe — 584, and zs; — 43 — x4 — O. 


P! P; P; P, P, Pl P, 8 
1 4 24 14 0 0 y 

0 -d — 246 — 24 1 0 224 

0 14 324 26 0 1 594 


We now have a basis of P;, P;, Ps, with P», P5, P, explicitly given in 
terms of these basis vectors, that is, 


SUUS h eo pue 
29P, — 414P,-- 3243P. — P, 
V$Pi x 24P; s 44P, —— P, 
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Hence if we wanted to obtain an extreme-point solution with P; in 
the basis, we could start with the second tableau and determine 6, as 
before and transform this tableau by the elimination formulas. "The 
resulting tableau will yield the representation of the vectors not in 
the basis in terms of the new basis vectors. 


EXERCISES 


1. Find all basic feasible solutions for the equations 


2r; 4- 625 4- 223 4- Xa mdi 
6x; JF 4x» 4d- 4x3 4- 67 — 2 


and determine the associated general convex combination of extreme-point solutions. 

2. Construct and graph linear-programming problems in three variables with a 
unique extreme-point optimum solution, three extreme-point optimum solutions, and 
four extreme-point optimum solutions. 

3. In the discussion in Sec. 3, the extreme-point solution (3.1) can be degenerate. 
Discuss the computation of 6 and the transformation to a new basie feasible solution 
if some z; — 0. Can there be any assurance that 6; 7 0? — Also discuss and interpret 
the situation where the selection of min (z;/z;;) for z;; » 0 is not unique. 


ü 
4. The following set of equations has a given extreme-point solution X — (ri,75,73) 
— (4,3,6). By the algebraic procedure of Sec. 3, obtain two basic solutions for the 
bases P;, P», P, and P;, P, P,. In each case, determine the expression for P; and 
the vector eliminated in terms of the new basis. 


P, P, P; P, P; P, 
T1 Jd 2-:4. — 3$ —4 
Xo — M4 4 X5 — 5 

Is -F3z4 — 225 zx 


b. Do Exercise 4 by setting up the tableau and applying the general formulas 
developed for the complete elimination procedure in Exercise 8 of Chap. 2. For 
both bases, the variable z, is eliminated from all the equations except one. 

6. For the 3 X 3 matrix formed by vectors P;, Ps, P, of Exercise 4 compute its 
inverse by means of the adjoint matrix as described in Chap. 2. Do the same for 
vectors P», P5, P.. 

7. Given the following set of equations: 


z;-- Ax. — Sy cmm 
5z, 4- 2x5 4- 323 —4 


Determine the basic feasible solution involving z; and zx». Do basic feasible solutions 


exist for z, and zs and for zs» and z;? ^ 
Discuss the graphical solution to this problem if each column of the matrix 1s 


assumed to be a vector in two-dimensional space. 


CHAPTER 4 / THE SIMPLEX 
COMPUTATIONAL PROCEDURE 


We shall next discuss and prove the validity of the basic elements of the 
simplex procedure and related computationalalgorithms. By the simplex 
procedure, we can, once any basic (extreme-point) feasible solution has 
been determined, obtain a minimum feasible solution in a finite number of 
steps. "These steps, or iterations, consist in finding a new feasible solution 
whose corresponding value of the objective function 1s less than the value 
of the objective function for the preceding solution. This process is 
continued until a minimum solution has been reached. From the discus- 
sion of Chap. 3, we have that all extreme-point solutions, and especially 
the minimum solution, have m linearly independent vectors associated 
with them. |. Wethen limit our search to those solutions that are generated 
by m linearly independent vectors. We note that there are a finite num- 
ber of such solutions. 


1l. DEVELOPMENT OF A MINIMUM FEASIBLE SOLUTION 


We assume that the linear-programming problem is feasible, that every 
basic feasible solution is nondegenerate, and that we are given a basic 
feasiblesolution.! "These assumptions, as will be discussed later, are made 
without any loss in generality. Let the given solution be X, — (1,220, 

. :£49)] and the associated set of linearly independent vectors be 
P,P,4...P, Wethen have 


TuPicE wuB.-Rot0*oRxPL— P (b) 
S101 -- X»o€2 -F ' "oc P Xmo€m 7 20 (1.2) 


! The definitions of these terms are given in Sec. 2 of Chap. 3. 
1 In order to generalize the simplex transformations we now denote the solution 
vector X — (xiz», . . .,2$4) by the veetor Xs — (Zi1o,226, . . . jf£40). We should 


note that the remaining n — m values of the solution vector have arbitrarily been set 
equal to zero. 
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where all zi; 7 0, the c; are the cost coefficients of the objective function, 
and 2o is the corresponding value of the objective function for the given 


solution. Since Pj, P, . . . , P, arelinearly independent, we can express 
any vector from the set Pj, Pj, . . . , B, in termsof P, Pj, .. ., P, 
Let P; be given by 

235 P3 Es 29;P» E E xr 4 2b: -— P; 4) —-— 1h ZEE RT (1:3) 
and define 

ducis d.n owost GP Nm. 99-2; ues POR e B9 (1.4) 


where the c; are the cost coefficients corresponding to the P;. 


Theorem 1. /f, for any fixed j, the condition 2; — c; 7» 0 holds, then a set 
of feasible solutions can be constructed such that z « zo for any member 
0f the set, where the lower bound of 2 is either finite or infinite. — (z is the 
value of the objective function for a particular member of the set of feasible 
solutions.) 

Case I. lf the lower bound is finite, a new feasible solution con- 
sisting of exactly m positive variables can be constructed whose value 
of the objective funetion is less than the value for the preceding 
solution. 

Case II. lf the lower bound is infinite, a new feasible solution 
consisting of exaetly m -- 1l positive variables can be constructed 
whose value of the objective function can be made arbitrarily small. 

The following analysis applies to the proof of both cases: 

Multiplying (1.3) by some number 6 and subtracting from (1.1), 
and similarly multiplying (1.4) by the same 6 and subtracting from 


(2p dTopg edge. Thee" 
(1o n 01,;) Pi "E (3:20 Fw 025;) P» uis E E (Q6 e) Ürz P. Bis 0P; 
(210 Sm 0:1;)€1 3E (3:20 io 01:2;)€2 c e es XT (3r Nee 02:6 zs 0cj 72 


— £y — 0(z; — cj) (1.6) 


where 6c; has been added to both sides of (1.6). Ifallthe coefficients 
of the vectors P;, Pj, . . . , P4, Bjin (1.5) are nonnegative, then we 
have determined a new feasible solution whose value of the objective 
functionis, by (1.6), 2 — zo — 0(z; — cj). Sincethe variables x10, 220, 
.. s , ao in (1.5) are all positive, it is clear, from our discussion in 
Sec. 3 of Chap.3, that thereis a value of 0 — 0 (either finite orinfinite) 
for which the coefficients of the vectors in (1.5) remain positive. 
From the assumption that, for a fixed j, 2; — c; » 0, we have 


Z — 29 — 0(z; — cj) « 2o 
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for » 0. We see that in either event a new feasible solution can be 
obtained whose corresponding value of the objective function is less 
than the value for the preceding solution. 

The proof of Case I follows: 

If, for the fixed j, at least one zi; » O in (1.3) for? — 1,2, . . . , 
m, the largest value of 0 for which all coefficients of (1.5) remain non- 
negative Is given by 


0, — min Z* $0 (1.7) 
"n 

for zi; 2» 0 (see Sec. 3 of Chap. 3). Since we assumed that the prob- 
lem is nondegenerate, i.e., that all basic feasible solutions have m 
positive elements, the minimum in (1.7) will be obtained for a unique 
i. If 0s is substituted for 0 in (1.5) and (1.6), the coefficient corre- 
sponding to this unique 2 will vanish. "We have then constructed a 
new basic feasible solution consisting of P; and m — 1 vectors of the 
original basis. "This new basis can be used as the previous one. 1f 
a new 2; — c; 7» 0 and a corresponding z;; 7 0, another solution can 
beobtained which has a smaller value of the objectivefunction. "This 
process will continue either until all 2; — c; X 0, or until, for some 
2,— 06; 0,all z; € 0. If all 2; — c; € 0, the process terminates. 

For Case II we have: 

If at any stage we have, for some j, 2; — c; » 0 and all zjj; € 0, 
then there is no upper bound to 0 and theobjective function has alower 
bound of — ». We see for this case that, for any 60 7 0,all the coeffhi- 
cients of (1.5) are positive. We then have a feasible solution con- 
sisting of m -F 1 positiveelements. Hence, by taking 6 large enough, 
the corresponding value of the objective function given by the right- 
hand side of (1.6) can be made arbitrarily small. 


Theorem 2. 7f for any basic feasible solution X — (xio,20, . . . ,t4o) the 
condileons 2; — 6; S 0 hold for all 4j. — 1,2, .. . , m, then (1D) amd 
(1.2) constitute a minimum feasible solution.! 


! This optimality criterion is sometimes varied. For a minimization problem we 
could have computed, instead of the z; — c;, the numbers c; — 2; and selected as the 
vector to be introduced into the basis the one corresponding to the min (c; — 2;). An 
optimum has been reached when all c; — 2; 7 0. If the problem was originally to 
be maximized, we could use the following criterion instead of changing to a minimiza- 
tion problem: Compute the 2; — c; and select a new vector corresponding to min 
(2; — cj); an optimum solution has been found when all z; — c; 2 0. Or compute 
the c; — z; elements with the new vector corresponding to max (c; — z;), the procedure 
stopping when all e; — 2; X 0. It is much more efficient, however, especially in 
developing a computational procedure for an electronic computer, to select one 
criterion to be used in solving all problems. $ee page 71 for the discussion on the 
eriterion used to select a vector to be introduced into the basis. 
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Proof. Let 
VioPi d- yaPa - ^55 dF yaPs — Po (1.8) 
and 
Uswcr o ysecrcIo c ot * b yas EU (1.9) 


be any other feasible solution with z* the corresponding value of the 
objeetive function. We shall show that zo X z*. (Note that the 
nondegeneracy assumption is not required for this theorem.) 

By hypothesis, 2; — c; X 0 for all j, so that replacing c; by 2; in 
(1.9) yields 


UnecitsdsUsgeuipo scs SE Uude Ss (1.10) 
For each j we substitute the expression for P; given by (1.3) into (1.8), 
to obtain 

y10 ( Y züP;) "- 1/20 ( x viPi) Motus 1/n0 ( » zisPi) 

i-c1 i-1 ici 
— P 
or, by regrouping terms, 

(D we) Pie» wis) Peto o wisis) Ps 

j-1 j-l eso 
SU D 


Similarly, for each j we substitute the expression for 2; given by (1.4) 
into (1.10) to obtain 


(5 Vjoti;) C1 xm e Vjota;) €» d- SEE (x Jjo*nj) Up Ss Sus 25 
j-1 j-1 
(12) 

Since the set of vectors Pj, Pj, . . . , B, is linearly independent, the 
coefficients of the corresponding vectors in (1.1) and (1.11) must be 
equal,! and hence (1.12) becomes 

T3061 -|- 42902 d - 77" E Xum SS ET 
Or by (2) 29 S e. 
'The results of Theorems 1 and 2 enable us to start with a basic feasible 


solution and generate a set of new basic feasible solutions that converge 
to the minimum solution or determine that. & cs solution does not exist. 


gence e to the minimum solution. Vertu T "ns assumption, de 


1 See footnote of Theorem 3, Chap. 3, for a proof of the equality of the coefficients 
of (1.1) and (1.11). 
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would be possible to have at least one of the.m.x;s-of the given solution 
equal to zero-—H Xhis-were-the case, then 69 could equal zero and the value 
of the objective function for the new solution would then equal the value 
of the old solution. "This lack of improvement in the solution could 
continue for a number of successive steps. For this situation, the proce- 
dure could conceivably repeat a basis and hence keep returning to this 
basis. Thesimplex procedure is then said to have cycled, and the compu- 
tational routine for determining the minimum solution breaks down. 
During actual computation, the phenomenon of degeneracy is reflected 
by basie solutions with less than m positive z;; and/or by more than one 2 
yielding 0, — min (z;o/z;j) for xi; » 0. If thez is not unique, some of the 


zio in the new solution will equal zero. 

Dantzig, Orden, and Wolfe [32] and Charnes, Cooper, and Henderson 
[12] have resolved degeneracy from both the theoretical and computa- 
tional points of view. Computational experience, however, does not 
warrant incorporating their "degeneracy techniques" into the standard 
simplex procedure. Out of the many linear-programming problems con- 
sidered by investigators in the field, only three have been known to cycle. 
These were artificially constructed by Hoffman (58] and Beale [4] to 
demonstrate that cycling could oceur. What is normally done in à com- 
putation is to treat degenerate solutions as nothing unusual and to com- 
pute with 0; — 0 whenever the procedure yields such a value for 0. If 
ties occur when determining 6,, the usual ruleis to select 0 — min (z;o/2) 


for the smallest index ?. Degeneracy techniques and a cycling problem 
of Beale are discussed in Chap. 7. 


2. COMPUTATIONAL PROCEDURE 


In this section we assume either that (1) we have selected m linearly 
independent vectors that yield a feasible solution and have expressed all 
other vectors in terms of this basis or that (2) our problem matrix con- 
tains m vectors that can be explicitly arranged to form a unit matrix of 
order m. 

For the first case let the m linearly independent vectors be P, P5, 

. ; P, and denote the m X m matrix (PP; - - - P,) by B. The 
matrix B is termed an admissible basis. 'To compute the corresponding 
solution vector X and the representation of the other vectors in terms of 
the basis, we must first compute B-!. Since 


BX, c P, 
we have 


Xo -— B-!P, 
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and 

X; — BP; 
where 

X» - (2:10,3:20, Semen Uno) 1) 0 
and 

Ao ex DES s ct Saas) 


are both column vectors. 
To start the simplex process, we group the vectors of the problem 
matrix as follows: 


(P, | PP; ap. P, | Peu DA e P, 
Or 
(Po BI D s B.) | (2.1) 


By multiplying the elements in the paritioned matrix (2.1) by B-!, we 
obtain 


499 0 D. cm jas se) 


Since we know the cj, we next compute the z; — c; and determine whether, 
for any j, the corresponding z; — c; » 0. If so, we carry out the com- 
putational procedure described in Theorem 1. If not, we have found the 
minimum feasible solution. In general, since we have no assurance that 
an arbitrary set of m vectors from the given set will be linearly inde- 
pendent, let alone yield a feasible solution, our assumption in the first 
case is not the usual situation encountered in practice. (There have 
been computational procedures proposed that do start with an arbitrary, 
but educated, selection of m vectors. Some of these procedures are 
deseribed in Sec. 1 of Chap. 9.) The condition of the second case 1s quite 
common, and we shall describe it in great detail. 
In thesecond ease we assume that the given set of n vectors Pj, . . . , 
P, contains m unit vectors that can be grouped together to form an m X m 
unit matrix. It will be shown in Sec. 3 below that this assumption 1s not 


restrictive. We let these vectors be P,, Pj, . . . , B, and take as our 
admissible basis 
B E (P;P * $9 5 P.) — T- 


Since B-! — L, and since all the elements of P, were originally assumed 
to be nonnegative, we have the initial extreme-point solution 


Xo E P, 
and 
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where 

Xo — (210,320, "Hw n E) X40 2s 0 
and 

X; -—- (215,255, Speer too) 


To start the simplex procedure, we arrange the problem matrix as 
shown in Tableau 4.1. (In practice, one does not have to group the unit 


Tableau 4.1. Initial step of computational procedure 


Zm,m-ol 


ZmeHl1 — €m41 


vectors together, but we shall do this for illustrative purposes.) From 
the original equations of the problem given by AX -— b, we have let 
zi; -— b;and xj — ag. zjforj —0,1, . . . , n is obtained by taking the 
inner product of the jth vector with the column vector labeled c, that is, 


Tn, 


gr Y CX io 


g-— Y cm da cap du ee 
1-1 
The elements 29 and 2; — c; are entered in the (m -- l)st row of their 
respective columns. "The 2; — cj for those vectors in the basis will always 
equalzero. lIfallthenumbersz; — c; € 0for; — 1,2, .. . ,n, then the 
solution X, — (xio22o, . . . 240) — (bj0s, . . . ,b,) isa minimum feasible 
solution, and the corresponding value of the objective funetion isz,.. We 
shall assume at least one 2; — c; ? 0 and compute a new feasible solution 
whose basis contains m. — 1 vectors of the original basis PP», . . . , P. 
In searching for à new vector to enter the basis, we can theoretically 
select any veetor whose corresponding 2; — cj » 0. As Dantzig [17] 
points out, the number of iterations necessary to obtain a minimum solu- 


Sec] THE SIMPLEX COMPUTATIONAL PROCEDURE 71 


tion ean, in general, be greatly reduced by not selecting at random any 
vector P; with its 2; — c; 7 0, but by selecting the one which gives the 
greatest immediate decrease in the value of the objective function. "The 
vector P; should then be the one which corresponds to the 


max 6o(z; — c;) (2.1a) 
J 


where, for each j, 0; is given by (1.7). If there are a number of j for which 
2; — cj 7» 0, the above rule is rather complicated to apply. A much 
simpler criterion for selecting the vector to be introduced is to select the 
one which corresponds to the 

max (z; — cj) (2.105) 

J 

If there are ties, the rule is to select the vector with the lowest (or the 
highest) index j. This criterion is the standard one employed in most 
computation centers and has proved to be an excellent one. When 
using this rule, approximately m changes of basis are required to go from 
the first feasible solution to the minimum solution. We shall employ the 
second eriterion, (2.15).! 

In our example, let 


max (eg —c)22,—c»0 
j 


The vector P, is to be introduced into the basis. We next compute 


for za » 0.[ If all zu X 0, we can then find a feasible solution whose 
value of the objective function can be made arbitrarily small (Theorem 
1, Case II). Our computation is then complete. Assume, however, 
some z;, » 0 and 

05 — min 2? Ec ur. 

Q0 0Tik X 

Vector P; will be the one eliminated from the basis. Our new feasible 
solution will have a new basis consisting of Pj, . . . , PL a Pa, ..., 
P,, P,.. We next wish to compute the new solution explicitly and to 
express each vector not in the basis in terms of the new basis. 

Since our initial basisis (P;P» : : - P,) — L,, wecan readily express 
all the vectors P; in terms of this basis. We then have 


1S8ee the work of Wolfe and Cutler [105g] and Quandt and Kuhn [85e] which 
describe and evaluate alternate selection rules. 

1 For the general situation the index ? ranges over those ? whose corresponding 
variables are in the basie solution. 
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P, — z19P1 Em dE -d- xioPi E S RES s Tob (2.2) 
p.—raoBi bans T ZskPm (2.3) 
and 
poepsgBy-c-cti «exuBpckbosctow UpjPm (2.4) 
From (2.3) 
Pp,-— E (P; Ee Pun Paese Pusiredes apa) (2.5) 
Ik 
Substituting the above expression for P; into (2.2), we obtain 
I 
PasaPpd cct E: (Pio ZuBg-—— ewe sP.) | 
XU 
T AM d Xosbu, 
Or 
P. (zo - Sou) B 9 cin qp us LE: 
Xu Ur 
RIS (s. a» x 2 Ps 
Tk 
"The new feasible solution X4 «€ (x45... 39 9. 32d), dug 20,18 
given by 
P -— zjPi d nsn c axP,- bs cr xs 
where 
UE E fori- 1,2 . yb EEREVCXm 
i (2.6) 
TE 
Ty Zu 


Similarly, by substituting (2.5) into (2.4), we can obtain the expression 
for each P; not in the new basis in terms of this basis. "This yields 


/ y /) 
P; L— ZuPI EZ ast s -- v,;P; d bv -— E 2S 
where 
dos " T 
X Xj — Zu for 7 »*1 
V 
(2.0) 
ji» cse geetli 
k3 Zu 
Since 
! — :h . . * f nA . . . La / 
2; — 6j — Ty6i d T-oxy6k d Sic uU 


T This expression is equivalent to Eq. (1.5) with j — k and 6 — zio/ziy — 6s. 
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one can readily verify by substituting the values (2.7) for 2 that 

/ -— TX, 

2; — 6j — £j — cj — — (ez — c) (2.7a) 


and by substituting the values (2.6) for z/, into 


£j — OX ck ccc od 6x ccc od 6n 
that 
; z 
ge r8 (2.7b) 
X 


We then note that, in order to obtain the new solution X;, the new 
vectors x6 and the corresponding z; — c,, every element in Tableau 4.1 
for rows? —1,...,m --1and columns j 2 0, 1, .. ., n is trans- 
formed by the formulas 


/ Tu 


(MT EAQUA fori zl 
Xu 
Xu € 
pu — (2.8) 
X 
where 
y -- [P , A / 
£9 — T441,0 &j — €j — T4415 


Here we are letting the general formulas (2.8) apply to all elements of the 
computational tableau including the P, column and the (m -4- 1)st row. 
'The transformation defined by (2.8) is equivalent to the complete elimina- 
tion formulas when the pivot element is zi. 

Once an initial computational tableau has been constructed, the 
simplex procedure calls for the successive application (1.e., an iteration) of: 


1. The testing of the 2; — c; elements to determine whether a minimum 
solution has been found, i.e., whether z; — c; € O0 for all j. 


t The reader should note that we are now letting the indices ? and [ refer to the cor- 
responding rows in the computational tableau. We keep track of which variables are 
in the current basic solution by means of the column labeled Basis. Hence, the 
general elimination formulas given by Eqs. (2.8) are applied to each element z;;, where 
irefers to the row and j to the column of thetableau. Element z;, is the pivot element 
of the elimination transformation, with row / being the pivot row and column £k being 
the pivot column. 

In applying Eqs. (2.8) to the tableau we can use two schemes. For manual com- 
putation it is best to compute first the new elements of the pivot row and add suitable 
multiples of this row to the other rows of the tableau in order to carry out the elimina- 
tion of z;o from all rows except the /th. For automatic computation by an electronic 
computer, it is best to compute first the ratios rj;,/ziy corresponding to the pivot 
column. The elimination transformation 1s then performed by applying Eqs. (2.8) to 
each column. Here it is probably best to carry the number 2; — c; as the first element 
of each column. 
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2. The selection of the vector to be introduced into the basis if some 
2; — cj 7» 0, i.e., selection of the vector with maximum 2; — Cj. 

3. The selection of the vector to be eliminated from the basis to ensure 
feasibility of the new solution, i.e., the vector with min (zio/zix) for 


those r4 2 0, where k corresponds to the vector selected in Step 2. 
If all z4 X 0, then the solution is unbounded. 

4. The transformation of the tableau by the complete elimination pro- 
cedure to obtain the new solution and associated elements. 


Each such iteration produces a new basic feasible solution, and by the 
discussions of Theorems 1 and 2 we shall eventually obtain a minimum 
solution or determine an unbounded solution. 

Àn application of the simplex procedure to the initial tableau yields 
the transformed values of Tableau 4.2. 


Tableau 4.2. Second step of computational procedure 


P»aa 


zu, mal 
z'», mal 


EST. mal 


, 
Im,mzsl 


, 
Z m4l — €m41 


Example. As an example, let us solve the following linear-programming 
problem by means of the simplex procedure: Minimize 


X2 — 3X3 c-r 2x5 
subject to 
ad) 
and 
q3- 345 — $$ jd- 225 z- 7m 
—2x» 4- 4x3 -- x4 p 
— 423 - 3x3 4- 8z5-L- s. — 10 
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Tableau 4.3 


Initial Step 


[S 


Ne» | N69 NE Ne 


PN 


Our initial basis (see Tableau 4.3) consists of P;, P,, Ps, and the 
corresponding solution 1$ X, — (xi2z4,s) — (7,12,10). Since 


CINCO) 


the corresponding value of the objective function, zo, equals zero. 
P; is selected to go into the basis, since 


max (2; —cj)) 223—063 —- 350 
j 
09 is the minimum of z;o/zis for zi; 7 0, that 1s, 
minui ae) o d mda 


and hence P, is eliminated. "We transform the tableau (see Second 
Step, Tableau 4.3) and obtain à new solution 


X Ls (2323,76) ES (10,3,1) 
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and the value of the objective function is —9. In the second step, 
since 

/ / 
max (2;— 65) 2,—0921$0 
j 


and 


- 
je! 


| 


0s — 


NO 
tON 


P» is introduced into the basis and P; is eliminated. We transform 
the second-step values of Tableau 4.3 and obtain the third solution 


XU m(assp$0 x (45,11) 
with a value of the objective function equal to —11. Since 
max (ez — cj) — 0 


this solution is a minimum feasiblesolution. Asacheck on each com- 
plete elimination transformation, one should explicitly compute the 
individual 29 and 2; — c; and compare them with the transformed 
values of 2; and 2; — cj. If for the minimum feasible solution some 
gj — n — 0 for a vector P; not in the final basis, then this vector can be 
introduced into the basis without changing the final value of the objec- 
tivefunction. "The resulting solution will also be a minimum feasible 
solution, and hence we have determined multiple minimum solutions. 
Any convex combination of these minimum solutions will also be a 
minimum solution. 

The determinants of the bases used in the simplex computation 
can be readily obtained as à by-product of the computation. Let 
the basis of the pth iteration be 


B,9(PBI- RUE) 
and the (p -- 1)st basis be 
B DB) 
where 
P, Eu » Cu P; 
Sel 
We see that 
il Zik 0 
Bip Bo -—BHibu c 
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where C, differs from the identity matrix in the th column. We 
then have 


[Bil -— IB, | : IC; zs ti [B] 


In the simplex process for p — 0, B, — I; hence the determinant of 
the pth basis of the simplex method is given by the product 


D 
[B,| L— I zy OG 


q-—0 


where z;,(? — 1, and x? are the pivot elements of the successive bases. 
We note that |B,| » 0. 


3. THE ARTIFICIAL-BASIS TECHNIQUE 


Up to this point, we have always assumed that the given linear-pro- 
gramming problem was feasible and contained a unit matrix that could 
be used for the initial basis. Although a correct formulation of a prob- 
lem will usually guarantee that the problem will be feasible, many prob- 
lems do not contain a unit matrix. For such problems, the method of the 
artificial basis (see Orden [84]) 1s a satisfactory way to start the simplex 
process. "This procedure also determines whether or not the problem has 
any feasible solutions. 
The general linear-programming problem is to minimize 


Puri RUN. Sud. 


subject to 
Unger iude da 0s 
ü»dqpp occ 3 iduots.s 
5I UVAWc ORUNURUUL. corr cur uro S 
and 
T; EU 


For the method of the artificial basis we augment the above system as 
follows: Minimize 


Cusyps:co Seda Esp co Wyss ccc dr Mun 
subject to 

üipbl cb o9 sas ^^ Sud —bi 

Goyty d» 0t FR 2n m — bs 


GOUOO  NCRUCHOEIOO IUUD TLDNEON SE REQUE UNE CHEE LME 1o SOEUR T RPEET TTC 1 TOC 


Gm T1 s vri se OmnYn ^E JZa-dm 7 Du 
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ku M Á |^ 


amddo cy 2 040p3 cx Tot romae e Sm The quantity w 


is taken to be an unspecified large positive number. The vectors P,,1, 
Pas .. . , País form a basis—an artificial basis—for the augmented 


system. If there is at least one feasible solution to the original problem, 
then this solution is also a feasible one for the augmented system. The 
simplex procedure will then ensure our obtaining the minimum solution, 
in which it is impossible for one of the artificial variables, z,.;, to appear 
with a positive value. If the original problem is not feasible, then the 
minimum feasible solution to the augmented problem will contain at least 
one z,4,; ^ 0. As we shall show below, it is not necessary to assign a 
specific value to w. 
For the augmented problem, the first feasible solution is 


X5 E (541,0, 0:870 vtr, inicio) c (b1,55, Ves D») 
T 
with a corresponding value of the objective function 29 — w 3» b;. Since 
íz1 
the basis is a unit matrix, X; — (xi,z5j . . . ,Z«;j) — (Qi509sj, . . . , Qj); 


Th 
ande; — w 2 zi. Aslong as there are artificial vectors in the basis, each 
1-1 


2; — cj will be a linear function of w. For the first solution, 
T" 
eg —— C 2 Xu 


. Each z; — c; will then have a w coefficient and a non-w coefficient which 
are independent of.each other. "We next set up the associated compu- 
tational procedure as Tableau 4.4. For each 7j, the non-w component 
and the w component of 2; — c; have been placed in the (m -- 1)st and 
(m 4- 2)nd rows, respectively, of that column. 


Tableau 4.4. First step of artificial-basis computational procedure 
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-We treat this tableau exactly like the original simplex tableau (Tab- 
.leau 4.1), except that the vectorintroduced into the basis is associated with 


the : largest positive element 1 in the (m -F 2)nd row. For the first itera- 


tion, the vector corresponding to max D ti; 1s Introduced into the basis. 
3 1 
The elements in the (m 4- 2)nd row are also transformed by the usual 


elimination procedure. Once an artificial vector is eliminated. from the. 


basis, 1t is never selected to reenter the basis. Hence we do not have to... 


"transform the last m columns of the tableau. If we are interested in the 
inverse of the final basis, theii these last m vectors should be transformed 


as usual! It should be noted that, even if there are artificial vectors in , - 


the basis, the iteration may not eliminate one of them. When using the 


full artificial basis, approximately 2m iterations are required to reach the | 


minimum feasible solution. 


Í 


| 


| 


.We continue to select a vector to be introduced into the basis, uSg 
the element D. the (m -- 2)nd row as eriterion, until either (1) all the... 


"artificial vectors are eliminated from the basis or (2) nó positive Qn 4 2)nd 
"element. exists. The first alternative implies'that all the elements in the 
(m 4- 2)nd row equalzero and that the corresponding basis is a feasible basis 
for the original problem. We than apply the regular simplex algorithm to 
determine the minimum feasible solution. In the second alternative, if 
the (m 4- 2, 0) element, i.e., the artificial part of the corresponding value 
of the objective function, is greater than zero, then the original problem 1s 
notfeasible. "Theorem 2 tells us that there are no other feasible solutions 
whose value of the objective function 1s smaller than this final solution. 
If the (m 4- 2, 0) element is equal to zero, then we have a degenerate 
feasible solution to the original problem which contains at least one arti- 
ficial veetor. "The artificial variables have values of zero. We have not, 
however, reached the minimum feasible solution. We continue the itera- 
tions by introducing.a. vector that corresponds to the maximum positivé 


element in the (m -- l)st row which is above a zero element in the* 


(m 4- 2)nd row.? This criterion is used until the minimum solution has 
been reached, i.e., until there are no more positive (m 4- 1)st elements 


over a zero in the (m -- 2)nd row. "The final solution may or may not. 


contain artificial variables with. values equal to zero. - When we use this 
criterion, the elements in the (m 4- 2)nd row are never transformed, since 
the 2; — c; of the vector being introduced is equal to 0 : w -F (e; — cj). 
For both alternatives (1) and (2), all the (m 4- 2, 7) elements are less 
than or equal to zero, with the possible exception of the (m 4- 2, 0) 


1 See Orden [84] for a discussion of the above and the application of the simplex 
procedure to related matrix problems. 

? This procedure forces the artificial variables still in the solution always to main- 
tain a value of zero and also eliminates the selection of certain vectors that would not 
reduce the value of the objective function. 


Í 
" 


l 
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element. 'The latter element is always nonnegative, and its value is non- 
inereasing. As Orden [84] points out, if the original problem is feasible 
but has an unbounded minimum, the method of the artificial basis will 
determine the feasibility before the unboundedness. 

Whenever the original problem contains some unit vectors, these 
veetors along with the necessary artificial ones should be used for the 
initial basis. Doing this will tend to decrease the total number of 
iterations. "The following will illustrate the artificial-basis technique: 


Example.! Maximize 


94-22. ^ 99g —— f4 


subject to 
X1 -4 225 4 3x3 -— 5 
2xi-- zx» 52x 9») 


Z21-- 22» -- zx. d x4 — 10 


and z; 2 0. Sinee we are always minimizing, the corresponding 
objective is to minimize —zi; — 2r» — 3zs 4- x4. 

Since the given system contains a unit vector P4, we need only 
two artifieial vectors P; and P, "The first solution (see Tableau 4.5) 
is Xo — (z;,2624) — (15,20,10), with a value of the objective function 
equal to 10 -- 35w. . Each 2;1s computed by taking the inner product 
of P; with the column vector c. For example, 


21-0 95 4D E Dur Toe (5 mE EU 


Vector P5is introduced into the basis because the maximum (m 4 2, j) 
element is (m -- 2, 3) — 8. The corresponding 0, — 296, and the 
artificial vector P; is eliminated from the basis. We transform all the 
elements of theinitial step of Tableau 4.5 by the elimination formulas. 
The new solution is X, — (r;z324) — (3,4,0), with a value of the 
objective funetion of —6 -4- 3w. P» goes into the basis, and P; is 
eliminated. In the third step, since all the (m 4- 2, j)) € 0 and 
(m 4- 2, 0) — 0, we have a feasible solution to the original problem; 
this solution is 


x x (12,23,24) z (194,254,154) 


! This simple example used to illustrate the computational technique also points 
out the advantages of applying a bit of analysis to the equations of a linear-program- 
ming model before solving it. Here, since z; -- 2x; 4-34 — 15, the problem is 
really to maximize the objective function 15 — z,. Hence, for z, » 0, the maximum 
basic feasible solution involves z;, zs, and z; with a maximum value of 15. 
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Tableau 4.5 


Initial Step 


ON | 2v Oy ON 
o |o v- 
[1i [^ 


NO Mt | N62 ND NI 


E 


m 
TÍN -IN MÍN 


NO | NO NOD NO 
m 


COSE 


with the objective function equalto —9972. "The fourth step yields a 
minimum feasible solution 


X ux (22,23,21) 7E ($5,9$,94) 


with z, — 0 and the objective function equal to —15. "The correct 
value of the objective function is 4-15, since we were originally dealing 
with a maximization problem. 
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4. A FIRST FEASIBLE SOLUTION USING SLACK VARIABLES' 


If the linear-programming problem is originally of the form AX X b, its 
corresponding initial computational tableau will contain an m XX m unit 
matrix. This matrix appears when we write each inequality as an equality 
by adding a nonnegative variable called a slack variable. 'These slack 
variables have associated cost coefficients that are usually set equal to 
Zero. 

For example, given the linear-programming problem: Maximize 


2x1 d- 32» 
subject to 
—z14d-2x; € 4 
Vise qa eb 
2i 944 € 9 
1,0 
Transforming to equations by adding a new nonnegative slack variable 


to all inequalities except the nonnegativity inequalities, we obtain the 
equivalent linear-programming problem: Maximize 


2z1 4- 32s 
subject to 
—321 - 223 -- 2s —4 
$4 7r 33 T 44 —6 
q1-- 3s dmg x-9 


3;20 


We see that these equations contain a starting basis of the slack vectors 
(P;P,P;) with the associated first feasible solution of rz; — 0, zs — O0, 
t$; — 4,1, — 6, 2; — 9. The.corresponding value of the objective func- 
tion is zero, as all cost coefficients for the slack variables are here assumed ^ 
to-be Zero. Some applications might call for these cost coefficients to be 
othér than zero, e.g., penalty costs for not using certain raw mmganrg 
storage costs. —— 
If the problem was originally AX 7 b, the equivalent set of equations 
is obtained by the subtraction of a new nonnegative slack variable from 
each inequality. Here the equations contain a negative unit basis which, 
since we assume b 7 O, cannot be used to initiate the computational pro- 
cedure. For this case the problem, in general, must employ at least one 


! See Sec. 4 of Chap. 2 for previous discussion of slack variables. 
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artificial vector. "This is further described in Sec. 1 of Chap. 9. A 
problem ean have a mixture of X, 7, and — constraints and is put into 
equation form by suitable additions or subtractions of slack variables. 


5. GEOMETRIC !NTERPRETATION OF THE SIMPLEX PROCEDURE 


The preceding algebraie description of the simplex procedure has the 
following geometrie interpretation, described in Hoffman, Mannos, 
Sokolowsky, and Wiegmann [61a]. It was first given in Dantzig [17]. 
Let the (m 4- 1)-dimensional vectors Pj, Pj, . . . , P; beobtained by 
appending the cost coefficients ci, €», . . . , c,, respectively, to the 
m-dimensional column vectors Pi, P», . . . , P,; that is, for P; the corre- 
sponding c; becomes the (m 4- 1)st coordinate. Let C be the convex cone 
in (m 4- 1)-space determined by Pj, Pj, . . . , P/.. Let B be the line in 
(m 4- l)-space consisting of all points whose first m coordinates are 
b, 053, .. . , b,. "This line can be thought of as being generated by an 
(m 4- 1)-dimensional vector P, whose first m coordinates are b, b;, 
... , 0, and whose (m -- 1)st coordinate takes on all values. 'The 
object of the computation is to find the lowest point of B which is also 
in C, that is, the point of B in C whose (m 4- 1)st coordinate is a minimum. 
The computational technique is as follows: Assume that m of the vec- 
tors Pj, P5, . . . , P;, say the first m, are linearly independent and have 
the property that the m-dimensional cone D which they form contains à 
point of B. "This property is equivalent to saying that the m vectors are 
an admissible basis, 1.e., correspond to a feasible solution. "These vectors 
can either be given, as in Sec. 2, or be artificial, as in Sec. 3. The hyper- 
plane containing D divides the remaining vectors P; into two groups. 
One group is on the side of the hyperplane which contains the positive 
(m 4- l)st coordinate axis. "The second group contains all the vectors 
which are below the hyperplane and are eligible to be selected as new 
basis vectors. Each vector of this second group can be joined to the 
hyperplane containing D by a line segment parallel to the (m -4- 1)st 
coordinate axis. Let P; be the vector with the property that its line 
segment is the longest. "This corresponds to selecting the vector with 
max (z; — cj) » 0. "Then P; and a certain set of m — 1 of the vectors 


p, 2. , E; have the property that the m-dimensional cone they form 
contains a point of B, and this point will be lower than the intersection 
of D with B. We replace the eliminated vector by P; and continue the 
process until a minimum solution has been found. 

We next illustrate the above with an example. Consider the problem 
of minimizing 


5zi 4d 322 4 433 4 6x4 
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subjeet to the conditions 


1 4 323 JJ 523 E 0x4 


Il 


621 zm 44» 4 2x3 EN Mam 2 
and 
We have 
1 3 5 6 
Pj-10 P,-|4 p.12 P,-|1 
D 3 4 6 


We plot these points as shown in Fig. 4.1. Since the line B intersects the 
cone C, feasible solutions to the problems exist. We can readily show 
that Pj and P are linearly independent and that P, can be expressed as à 


»a0; 


FIGURE 4.1 


positive combination of P; and P... We then take as our first basis vectors 
P; and P,, and we determine the two-dimensional cone formed by vectors 
P; and Pj (Fig. 4.2). "The point P; is below the hyperplane containing 
D by a distance of 2», where 2; 1s defined in Eq. (1.4). P;is a distance 
2; below this hyperplane. Point P$ is below the hyperplane containing 
D by a distance of 2; — c». The corresponding value of the objective 
funetion is given by z. Since 2» — cs 7 23 — cs, we introduce P; into the 
basis. From Fig. 4.2 we see that P, can be expressed as a positive com- 
bination of vectors P; and P, and cannot be expressed as a positive com- 
bination of P» and P; that is, Po is not contained in the cone generated 
by vectors P, and P». Hence P; is introduced into the basis and elimi- 
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y FIGURE 4.2 


nates P;, giving a new basis of P» and P... For this basis we can perform a 
similar analysis, which will introduce P; into the basis and eliminate P,. 
'This basis of vectors P; and P; corresponds to the minimum solution. 

We may also interpret the simplex procedure in terms of moving from 
one extreme point to an adjacent extreme point (see Sec. 4 of Chap. 2 
and Saaty [88]. 'The procedure starts with the linear form passing 
through an extreme point of K (Fig. 4.3). Here we have pictured an 
arbitrary two-dimensional set of linear inequalities and its corresponding 
region of solution. 

The next iteration would advance the linear form parallel to itself 
until it passes through X;. In two additional iterations the linear form 
would pass through X;, and we would have the minimum solution. The 


x2 


FIGURE 4.3 
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total number of iterations necessary to reach the minimum depends on 
which feasible solution is used to initiate the computation. If, as above, 
we started with the solution that corresponds to the extreme point X, 
then it would take us three iterations. If the solution associated with 
X; were taken as the first, then it would require only one iteration. f 
the coefficients of the linear form and the equations are normalized, then 
z is the distance of the linear form from the origin. 


EXERCISES 


1. Solve the following linear-programming problems by the simplex method: 
a. Minimize 


$1. -1- 02 -[- 33 
subject to 
Zi — 44 — 2x4 — 
X2 4- 2-4, — 325 d- 2&3 


r3; 4 2134 — 525 4- 0234 — 5 
and z; 2 O0. 


b. Minimize 


EXE c RUE ES ere cM ue sa 
subject to 

T; EE T60r;,29 

3a; d zx» — 4z3 422422 

qi 4- 2z3 €. 25$ — 6 
and z; 2 0. 


c. Minimize 
2xi-- 3» — z.$— x4 
subject to 
qu mo cg — pic 
2x; JF x. — 3x3 d- z4 
Sy -E cb X354 
and z; 2 0. 


ll 


d. Minimize 
—3z s X2 1 3925 CR 
subject to 
zi - 227; — zd z,20 
2xi — 223 4-324 4-32, — 9 
Z1— 22--22; — 242 6 


and z; 2 0. 


Sec25] THE SIMPLEX COMPUTATIONAL PROCEDURE 87 


e. Maximize 


X4 — Z5 
subject to 
2xs — rà — 34,4 2:5, 0 
—2z, Cc 2zx3— 24-2420 
Xi — 2z5 819059600 
Use 3 ae ters —1 
and z; 2 0. 


f. Maximize 


Zi — $2 b za — Oz, d 3; — rg — 29m 


subject to 
3X3 cpu e dd —- 6 
zr; -LF 2x3; — d, zlü) 
to P o -— 
Xa es s Dp 
and z; — O0. 


g. Minimize 
qu 2r» 4 3z3 
subject to 


— 2x, 4l Zo -F 323 9) 
271 -l 3z5 -F 4zs zu 


and z; 2 O. 
À. Maximize 
DOT 
subject to 


2mm) 
z, T 34 € 3 
z; X 4 


nnde o0: 
i. Minimize 


2n exem 3x3 4 625 


subject to 
3z; — 4x9 — 0x3 € 2 
2z; - z$2 4-225 2 1l 
$;-4- 325 — 2z$ X 5 


and z; 2 0. 
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j. Maximize 


£1 -L t» 
subject to 
qM 
231—254 €1 
—42j1-d- 25 € 1 
and z; 2 0. 


2. Given a linear-programming problem that has a finite minimum feasible solu- 
tion. Let the minimum value of the objective function be z;. The problem is being 
solved by the simplex procedure, and in the kth iteration a degenerate basic feasible 
solution with exactly one z; — O0 appears. "The corresponding value of the objective 
funetion is Z with Z » zy. Prove that this kth basis cannot reappear in the succeed- 
ing iterations. 

3. Solve Exercise 1c by selecting P;, P», and P; as an initial basis; similarly, solve 
Exercise le with P;, P», P;, and P, as an initial basis. | Also compute the determinant 
of the final basis. 

4. (A test for a near-optimum solution in the simplex procedure due to L. Gold- 
stein.) Given a linear-programming problem with a known upper bound S to the sum 
of the variables, e.g., the transportation problem. Let 2 be the value of the objective 
function for a given basic feasible solution and z; be the unknown minimum value of 
theobjective function. Let — zy — c, — max (z; — cj) for the given basic solution. 
Show that Z — zy € e, 1s à predetermined tolerance limit, 1f m X e/S. 

b. Solve Exercise 1b, 1d, and 1? using the criterion (2.1a) for selecting the variable 
to be introduced into the solution. 

6. Prove the validity of the transformation formulas (2.7a) and (2.75). 

7. Prove the following statement: If the original problem is feasible but has an 
unbounded minimum, the method of the artificial basis will determine the feasibility 
before the unboundedness. 

8. Prove that if a set of m linear equations in n unknowns, where m X n, has a 
feasible solution, there exists a basic feasible solution. (Assume that the rank of the 
system ls m.) 

9. For the standard linear-programming problem, let 


Z9 — Cui CuEa db co coc db €aXa 


Write an equivalent linear-programming problem which uses this equation as a 
constraint and discuss the application of the simplex algorithm to the new problem. 

10. For Fig. 4.2 show that for some P; not in the basis, e.g., vector Ps, the 
corresponding z; does measure the vertical distance of the point P; from the plane D. 


CHAPTER 5 / THE DUALITY PROBLEMS 
OF LINEAR PROGRAMMING 


Associated with every linear-programming problem as defined in Sec. 1 
of Chap. 4 is à corresponding optimization problem called the dual 
problem. "The original problem is termed the primal. 'The optimum 
solution of either problem reveals information concerning the optimum 
solution of the other. In fact, if the initial simplex tableau for the primal 
problem contains an m X m unit matrix, then the solution of either 
problem by the simplex procedure yields an explicit solution to the other. 
We shall state and prove the theorems associated with the dual problems 
as given by Dantzig and Orden [31]. For additional reading, the reader 
is referred to Gale, Kuhn, and Tucker [45], Vajda [97], and Kuhn and 
Tucker [68]. 


1. THE UNSYMMETRIC PRIMAL-DUAL PROBLEMS 


The primal problem.! Find a column vector X — (xis, . . . ,2,) which 
minimizes the hinear functional 


f(X) 5 cX (1:5) 
subject to the conditions 
AX — b (1:2) 
and 
X20 (1.3) 


'This statement of the general linear-programming problem assumes that 
the number of rows of A is less than the number of columns of A. 

! For discussion purposes we have designated the minimization problem as the 
primal. In general, either problem can be considered the primal, with the remaining 
problem the dual. 


90 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 5 


The dual Problem. Find a row vector W — (wiws, . . . jw») which 
mazaimizes the linear functional 


g(W) — Wb (1.4) 


subject to the conditions 


WA zc (1:5) 
In the dual problem, the variables w; are not restricted to be nonnegative. 
For both problems we have that c — (ci€s, . . . ,C&) 1$ a row vector, 
b — (bibo, . . . ,b,) is à column vector, and A — (aij) is the matrix of 


coefficients. Multiplying the 1 X m row vector W by the m X n matrix 
A, we have the following explicit representation of (1.5): 


G11U; d- ds1€2 E ^ c 7 dT Gsm I 
Qi9W; d- da9Ws2 d- * * " -F Ars S Ca 
(1501 -- (254102 -r RE E Gong «x Cn 


The matrix of coefficients for the above inequalities 1s given by A'. 
Associated with these problems is the following important theorem: 


The duality theorem. 7f either the primal or the dual problem has a finite 
optimum solution, then the other problem has a finite optimum solution 
and the extremes of the linear functions are equal, that is, min f(X) — max 
g(W). 

If either problem has an unbounded optimum solution, then the 
olher problem has no feasible solutions.! 


Proof. We first assume that the primal is feasible and that a minimum 
feasible solution has been obtained by the simplex procedure. For 
discussion purposes, let the first m vectors Pj, P5, . . . , P, be the 
final basis vectors. Let B equal the m X m matrix (BjP, : : * P,). 
The final computational tableau contains the vectors of the original 
system, P, Pj, .. . , P5, P444 .. . , B, in terms of the final basis 
vectors; 1.e., for each vector P; the final tableau contains the vector 
X, where P; - BX, Let the m X n matrix 


Nu p.49 c EC do quM UN 


be the matrix of coefficients contained in the final simplex tableau. 
Since we assumed that the final basis contained the first m vectors, 


! For the dual problem, a solution is feasible if it satisfies (1.5). The variables 
are not required to be nonnegative. 
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we have 
0 n 0 LT mul cU] VES 
z H 0 1 E. 0 22, »ux ING 2n 
0 0 Er i Xa; mel Co 7— Xmn 


The minimum-solution vector is given by X? — B-!b. For this final 
solution we then have the following relationships: 


A — BX B-:A-X (1.6) 
be BX*  B-beX* (1.7) 
min c (1.8) 
Z -cX-—cxO (1.9) 

where c? — (eyes . . . cs) is a row vector and, by (1.4) of Chap. 4, 
Z — (c, — c, c?X5, — e, .. . , c?X, — c) 


— (£1-— €1, 2» — €2, . . . , £4 — €) 


is à row vector whose elements are nonpositive, as they are the 
2; — c; elements corresponding to an optimal solution. Here the 
vector O is a null n-dimensional row vector. 

Let W* — (w)w), .. . ,w2) be defined by 


W9 — co? B-1 
Then, by (1.6) and (1.9), we have 


W*'A—c-cB-A—c-cX—c«oO 
or 
W?A Cc 


The vector W? is a solution to the dual problem, since it satisfies the 
dual constraints (1.5). For this solution the corresponding value of 
the dual objective funetion (1.4) 1s given by W?b, or, from (1.7) and 
(1.8), we have 


W'b — c?*B-!b — c?X? — min f(X) (1.10) 


Hence, for the solution W?, the value of the dual objective function is 
equal to the minimum value of the primal objective function. Now 
we need only to show that W^? is also an optimum solution for the 
dual problem. 

For any n X 1 vector X which satisfies (1.2) and (1.3) and any 
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] X m vector W satisfying (1.5), we have 


WAX — Wb - g(W) (LED) 
and 

WAX € cX - f(X) (1.12) 
By (1.11) and (1.12), 

g(W) € f(X) (1.13) 


for all feasible W and X. The extreme values of (1.1) and (1.4) are 
related by 


max g(W) € min f(X) (1.14) 
For the dual solution vector W?, we have from (1.10) 
g(W?) — W?b — min f(X) 


and hence, for the optimum solution to the primal problem, X?, and 
the dual solution W?, (1.14) becomes 


g(W?) —f(X^) 


i.e., for the solution W? — c?B-!, the dual objective function takes on 
its maximum value. "Therefore, for W? and X? we have the corre- 
sponding values of the objective functions related by 


max g(W) — min f(X) CE 


The above results have been shown to hold whenever the primal 
hasafiniteoptimumsolution. Inasimilar fashion, we can show that, 
when the dual problem has a finite optimum solution, the primal is 
feasible, and (1.15) holds. 

To do this, transform the dual problem 

max g(W) 2 max Wb 

WA €Xc 
to the primal format and show that its dual (which will be feasible 
by the above theorem) is just the original primal. We have 


max Wb — — min(— Wb) 
Or 
— max Wb — min (— Wb) 


subject to 
WA--Wi-c 
W;20 
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where the W; is a set of nonnegative slack variables. We next 
transform W — W, — W;, where W; and W, are nonnegative varia- 
bles. Our problem is now 


min (— Wi; -- Wi)b 4- W;,O (1.15a) 
subject to 
(W;— WA -Wi-c (1.150) 
W,20 
W.20 
W;20 


and is in the standard format. 

'The dualitv theorem holds for this problem. To determine its 
dual in the proper format, 1t is convenient to multiply the constraint 
equation (1.155) by —1 to obtain 


— WA 4- W,À — Wil — —c (1:156) 
'The dual to (1.15a) and (1.15c) 1s then 

max (—cX) — —min cX 
subject to 


(-A A -DX-x(-b b O) 
which is equivalent to 
—max (— cX) — min cX 
subject to 
—AX «X —b 
AX X b 


—IX X O 
Or 
AX 2b 


AX Cb 


x cst) 
Or 
AX — b 


X20 


which is the original primal problem. 
This completes the proof of the first part of the theorem. 
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To prove the second part, we note that, if the primal is 
unbounded, then we have by (1.13) 


g(W) € —« (1.16) 


Anv solution to the dual inequalities (1.5) must have a corresponding 
value for the dual objective function (1.4) which is a lower bound for 
the primal objective function. Since this contradicts the assumption 
of unboundedness, we must conclude that there are no solutions to 
the dual problem and hence that the dual inequalities are inconsistent. 
A similar argument will show that, when the dual has an unbounded 
solution, the primal has no solutions. 


Example. As an example of unsymmetric dual problems, let us look at 


the example that was solved by the simplex procedure in Sec. 2 of 


Chap. 4. 
The primal problem. |. Minimize 
X» — 9x3 X 2x; 
subject to 
zx, 3x2. — T3 -- 2xg -— fí 
— 2x5 4- 4x; o - 24 — 12 
— Azo E 3x3 4 825 d- X8 — 10 
and 
X; 22 0 
Here we have c — (0,1, —3,0,2,0), b — (7,12,10), 
ji 2: Y 0. 2. 
ÀA-[|l0 —2 dp ood 
0 —4 9 0 8 1 
and 
1 0 0 
3 —2 —4 
—1 4 3 
A's 
0 1 0 
2 0 8 
0 0 1 


The dual problem. Maximize 


Tw 4-12w; Es 10203 
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subJect to 
W1 xU 
àwi|— 2w; — Aw 1 
—Ww;i-F A4ws-F 3w; € —3 
U» E) 
2wi RR UV 
wasc 0 


The final basis corresponding to an optimal solution to the primal 
problem, as computed by the simplex method in Sec. 2 of Chap. 4, is 
given by the vectors P», P;, P,, that is, 


SES ERIS) 
B - (P;P;P,) -— —2 4 0 
zi 3 1 


'The corresponding optimum solution X? is 
Xm Bb c1 25 bb 1) 
C? — (05,05,0ce) — (1,—3,0) 
and the minimum value of the objective function is 


4 
cx? - (,530[| 5|]2-11 
11 


From the final simplex tableau (see Tableau 4.3) we have 


26 10 Mo 460 
zx meceusmqicmope 20e qc 09 0) 
120:0- 5-350100 
Or 
Z- («50,0465 — 522,0) x O 


'The elements of Z are just those elements contained 1n the (m 4- 1)st 
row of the final tableau, i.e., the 2; — c; elements. 

In the discussion of the elimination method described in Sec. 5 
of Chap. 2, we showed that, if the original matrix of coefficients for 
the equations contained a unit matrix or was augmented by a unit 


96 


METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 5 


matrix, then the solution of the equations by the elimination method 
also yielded the inverse of the basis that corresponds to the solution. 
The elimination procedure transformed the unit matrix into the 
desired inverse. The same property is true for a linear-programming 
problem that has been solved by the simplex procedure. If the 
original matrix of coefficients, A, contains à unit matrix or is aug- 
mented by a unit matrix, then the inverse of the basis at each step 
is developed in the columns that correspond to the unit matrix. In 
our example, we see that A does contain a unit matrix whose columns 
correspond to P;, P4, P, Hence, in our final tableau, which is the 
third step of the example, the columns which correspond to P;, P4, P; 
have been transformed to the inverse of the final basis B. These 
columns form the set of vectors X;, X,, X;, and hence 


As we have shown in the proof of the duality theorem, the optimum 
solution W? to the dual is given by 


25 2h00 
W*-cB'-(1—30)|l 3(9 0]-2 (—16,—46,0) 
1-5 id 


We check this solution by substituting it in the dual constraints, and 
we have 


WA «cc 
T 
po vedo 9 ! 
(exem so cen es 
EM : 
0 

—-M tix 0 

is 

coro 

-* x 

-A x 
(e 
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We have as the value of the dual objective function 


The values of the variables for the optimum solution to the dual 
do not have to be obtained by multiplying c?^B-! if the matrix À con- 
tains a unit matrix. We have 


W? z cB-! zz cX;X,X&) z (w992,w) 


By definition of 2; we have c"X, — 2; CX, — 24, and cC"X, — 2. 
In the (m -- 1)st row of the final tableau we have for each X; the 
corresponding 2; — c; element. For j — 1, 4, 6 we note that the 


corresponding c; — 0, and hence the elements in the (m -4- 1)st row ." ; 
corresponding to j — 1, 4, 6 are equal to the corresponding values of 
the dual variables, that is, w? — z,, w9 — z, and w!-2, Jlfag 


vector which formed the unit matrix had a c; z& 0, then the value of 
this c; would have to be added back to the corresponding 2; — cj in 
the final tableau in order to obtain the correct value for the w?. We 
note that w? is equal to the 2; which has, for its corresponding unit 


vector in the initial simplex tableau, the veetor whose unit element 


is in position 7. In our example, w$ — 24, since P, is a unit vector 
with its unit element in position 7 — 2. 

A variation of the unsymmetric dual problems is given in Sec. 2. 
For both of these new problems, the symmetric dual problems, the 
constraints are inequalities and the variables are restricted to be 
nonnegative. We show that the duality theorem holds for the 
symmetric dual problems by transforming them to their equivalent 
unsymmetrie primal and dual problems. 


2. THE SYMMETRIC PRIMAL-DUAL PROBLEMS 


The primal problem. Find a vector X — (xis, . . . ,z,) which mzni- 
mizes the linear functional 
f(X) 2 cX (2.1) 


subject to the conditions 


AX ^ b 2/2) 
and 
X 0 (2:3) 
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The dual problem. Find a vector W — (wiws, .. . ,w») which 
mazimizes the linear functional 
g(W) — Wb (2.4) 
subject to the conditions 
WA xc (2:5) 
and 
wW O0 (2.6) 


We next show that the duality theorem of Sec. 1 also applies to the 
symmetric dual problems. Let the m nonnegative elements of the 
column vector Y — (yi,ys, . . . ,y») be the slack variables which trans- 
form the primal into a set of equations. The equivalent linear-program- 
ming problem in terms of partitioned matrices 1s to minimize 


-of[X 

ja: - 10» (Y) (2.7) 

subjeet to the constraints 
X 

(A | —I) (v) zb (2.8) 
and 

X20 and yc 0m (2.9) 
Here O — (0, . . . ,0) is an m-component row vector and I is an m X m 


identity matrix. 
The dual of this transformed primal is to find à vector W which 
maximizes 


g(W) — Wb (2.10) 
subject to the constraints 
W(A| — D) € (c| 0) RUP 
We see that (2.11) decomposes to (2.5) and (2.6), respectively, i.e., 
WA €c 
and 
-WI-xO 


The last expression is equivalent to W  O. 

The original problems are now given as unsymmetric dual problems, 
and the duality theorem holds. 

We next prove, for the symmetric dual problems, the following theo- 
rem which ean be interpreted as establishing necessary and sufficient 
conditions for the optimality of feasible solutions to the primal and dual 
problems. 


T See Sec. 5 of Chap. 2 for a discussion of the use of partitioned matrices. 
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Complementary slackness theorem (Dantzig and Orden [31a] and Dantzig 


[16c]). For optimal feasible solutions of the primal and dual systems, j^ r 
whenever inequality occurs in the kth relation of either system (the cor- 4^ 


responding slack variable is positive), then the kth variable of its dual 
vanishes; 1f the kth variable is positive in either system, the kth relation 
of its dual is equality (the corresponding slack variable 4s zero). 


Proof. Werewrite theinequalities (2.2) of the primal system as equalities 


by subtracting a nonnegative slack variable z,,; from the ith 1^ 


inequality to obtain 


Q1111 4 reds E QinX5s — Una b, 


Om 1 —s dnm "E OmnXn VIS Un-4-m -— D 


Similarly, for the dual inequalities (2.5) we add a nonnegative slack 
variable w,4; to the jth inequality to obtain 


drip 500 9E ais: a1 ECT 
(2.5) 
GQi4Wi E cocco Os d Wsys 7 Cn 
In explicit form, the primal objective function is 
€3231 d- 7 7 4F 6,24 — f(X) (2:17) 
and the dual objective function is 
biwi d cc c d bw — g(W) (2.4^) 


We next multiply the ith equation of (2.2) by the corresponding 
dual variable 2; ? — 1,2, . . . , m, add the resulting set of equations, 
and subtract this sum from (2.1/) to obtain 


7T T 
(& — Y auw) x - (e — Y, aawi) 8 cc 
Sed iz 


7n 
4 (o- 2 05) 9. Tibor orbes cde ce 


1-1 
d Usxsus — J(X) — Y, wb; 
i-i 
Noting that w44; — c; — n à;; w; and g(W) — P wib;,we have 
i-1 i-— 
UU m-4-1T1 ZI AU m 4-222 -- Ra -J IU m--nXn 4 1254-1 E 1 935.12 d 


"e c Wafuam 400) —90W) (2.12) 


4 


US 


j 


rd [ 
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From the duality theorem, we have for an optimum solution 
X? — (x9, x9, .. . , x2) to the primal problem and 


WU sep dS o. MR) 


an optimum solution to the dual problem that f(X?^) — g(W?) — 0. 
TRU for these THES solutions and corresponding slack variables 


z0,; 2 0 and w;,,; 2 0, (2.12) becomes 


um a [META Mes Fowe qm n ol w eR p w MS sp 
im WX E xii —0 (2.13) 


We note that the terms w;,, 7; are the product of the jth slack variable 
of the dual and the jth variable of the original primal; while the terms 
w;z$,, are the product of the ith variable of the original dual and the 
ith slack variable of the primal. Since all variables are restricted to 
be nonnegative, all the product terms of (2.13) are nonnegative and, 
as the sum of these terms must be ios to zero, they Tad vi uai 
must be equal to zero. "Thus w;,,;z ; — 0 for n and w?z?,, — 0 for 
alli. If w2,, » 0, we must have z; — 0; if z7,, 2 0, then w; — 0, 
which establishes the first part of the theorem. If z? » 0, then 
w$, — 0, if wi » 0, then z2,, — 0, whieh completes the proof. 


m-4-k 


In a shghtly different form, the basic duality theorem can be restated 
as follows (Dantzig [16c]) : If feasible solutions to both the primal and dual 
syslems exist, there exists an optimum solution to both systems and 


min f(X) — max g(W). 


We note that each member of a pair of primal and dual problems can 
have finite optimum solutions; that one problem ean have an unbounded 
optimum feasible solution while the other has no feasible solutions; and 
that both problems ean have no feasible solutions. 

As noted in Goldman and Tucker [51], the symmetrie dual problems 
can be conveniently represented by the following tableau: 


c0) 21 QUOND Dro DE Ue z ? 
wi Qi r1. 005  cnoos om bi v jo 
Un Q1 i; Qi, b, lr 
Un QGmi QGmj Gmn hrs 
zx ei Cj Cn max 
min 


By taking inner products of the rows of A with the row of z's, we obtain 
the constraints of the primal; the inner produets of the columns of A with 
the column of w's yield the dual constraints. 
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Many linear-programming problems are originally stated in the form 
of either the symmetric primal or the symmetrie dual. In either case, the 
problem is usually rewritten in equation form by adding or subtracting 
the necessary nonnegative slack variables. For the symmetric primal we 
subtraet a set of slack vectors and solve the corresponding system, while 
for the symmetric dual we add a set of slaek vectors. "These slack vectors 
are given associated cost coefficients that are equal to zero. "The resulting 
tableau for the primal contains a negative unit matrix, while the dual 
tableau eontains a positive unit matrix. Hence the final simplex tableau 
of either will contain the optimal solution of the other. If the primal 
was solved to obtain the dual solution, then, since we start out with a 
negative unit matrix in the primal, the corresponding z; values in the final 
tableau have to have their signs reversed. 

As the size of a linear-programming problem that can be solved on an 
electronic computer is usually limited by the number of rows involved, a 
problem that is too large when stated in terms of the primal may be of the 
rght dimensions when written as the corresponding dual problem, and 
vice versa. In either situation the optimum solution to the original prob- 
lem is contained in the final tableau. 

By means of the symmetric dual relationships the general linear-pro- 
gramming problem can be represented as a problem that is concerned 
with the solution of a set of inequalities 1n nonnegative variables. A 
vector X which satisfies the constraints 


AX 2— b 
WA -€c 
Wb 2 cX 
2500 
W20 


will be an optimum solution to the problem of minimizing 
cX 
subject to 
AX —b 
X20 


Let us briefly investigate the economic interpretation of the activity- 
analysis problem of Sec. 2, Chap. 1, and its dual.! "The primal problem 


1! See Allen [2], Baumol [3f], Dorfman [39] Dorfman, Samuelson, and Solow [40], 
Harrison [53], and Tucker [95] for further discussion. For an interpretation of the 
primal and dual problems (and linear programming in general) in terms of classical 
Lagrange multipliers, see Tucker [96]. 
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can be written so as to maximize 
cX 
subject to 
AX X b 
x 
where the a;; represent the number of units of resource 7 required to pro- 
duce one unit of commodity j, the b; represent the maximum number of 
units of resource 7 available, and the c; represent the value (profit) per 
unit of commodity j produced. "The corresponding dual problem is to 
minimize 
Wb 
subject to 
WA c 
W20O 
Whereas the physical interpretation of the primal is straightforward, 
the corresponding interpretation of the dualis not so evident. Questions 
arise as to the meaning of the dual objective function and inequalities. 
These can best be answered by interpreting the primal and dual problems 
in terms of physical units in order to determine the meaning of the dual 


variables (Harrison [53]). 
The primal problem is to maximize 


c value 
2 (oec ;) (output 7) 
- 


subject to 


2, (e) (output 7) € (input 2) DN UP Apul NES 


(value) 


LA output j 
ae 
(output 7) 2 0 VLL P NET 
and the dual is to minimize 
* (input 2)w; — (?) 
ici1 


subject to 


v / input i value 
erc uoce jou: 9 
e E ;) $m (Ae ;) q es qa UE 


w; 20 peeled CH 
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We see that the dual constraints will be consistent if the w; are in units 
of value per unitofinput?. 'Thedual problem would then be to minimize 


y (input 2) (oou ) — (value) 


input 2 

subject to 
y input P value. - value Due seruis 
m output j/ Xinput 2 output 


(ame; zo $21,2,...,m 
input 2 


Verbal descriptions of the primal and dual problems ean then be stated 
as follows: 

The primal problem. With a given unit of value of each output (cj) 
and a given upper limit for the availability of each input (bj), how much 
of each output (z;) should be produced in order to maximize the value of 
the total output? 

The dual problem. . With a given availability of each input (b;) and a 
given lower limit of unit value for each output (cj), what unit values 
should be assigned to each input (10;) in order to minimize the value of the 
totalinput? The variables w; are referred to by various names, e.g., 
accounting, fictitious, or shadow prices. 

In line with the above, let us assume that we have determined by 
means of the simplex method an optimum solution to the activity-analysis 
problem. As we are maximizing the objective function, all vectors both in 
and out of the basis will have an associated z; — c; 2 0. Because of 
some external conditions, e.g., government regulations or marketing con- 
ditions, the problem might stipulate that the kth activity must be in the 
final solution at a positive level. However, the optimal basis does not 
include vector Pj. We must then alter our maximum solution by forcing 
vector P, into the basis and removing some other activity. Assuming 
that P, does not generate a multiple solution, i.e., that we have z, — cy 7 0, 
the introduction of P, reduces the maximum value of the objective func- 
tion by the amount (z; — c;)0s. This value is the loss in profit forced 
upon the manufacturer by the external condition. "The value of z; — c; 
is the net cost of introducing one unit of the kth activity, where the z, 1s 
termed the indirect cost and the c, the direct cost. "The relationship 
between the net costs and accounting prices can be seen by analyzing the 
simplex tableau corresponding to the optimal solution of the activity- 
analysis problem. — The set of indirect costs, i.e., the z; — c; elements that 
correspond to the slack vectors, represents a set of accounting prices that 
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solve the dual problem. If all the slack variables are in the final basis, 
then, in terms of the primal, no outputs are generated, no inputs are used, 
and the maximum profit is zero. For the dual this means that the mini- 
mum value of the total input is also zero. "This is evidenced by all the 
indirect costs corresponding to the slack vectors being equal to zero.! 
If the optimum solution of the primal does not involve any of the slack 
variables at a positive level, then the primal problem yields à positive 
profit while utilizing all the available inputs. For this ease none of the 
indirect eosts associated with the slack vectors is equal to zero, and the 
minimum value of the total input is positive and equal to the maximum 
profit. 


'EXERCISES 


1. Write out the corresponding primal problem to the following symmetric dual 
problem and solve both problems by the simplex procedure: Maximize 


Ww; d ws 4d ws 
subject to 


2w; 4c W»--2wi € 2 


4w; c 2ws -- wi € 2 
Wi 1) 
WW» 22210, 
w3;20 


2. Solve the following problem by the simplex method: Minimize 
22; m 32x» 
subject to 
2x — wg. 23 3 
Bus Vno rp eco do 
T1 z 0 
0 
Xs 2 0 


S 
IV 


3. Write out the dual problem to the problem given in Exercise 2 and solve it by 
graphing the constraints. 


4. Graph the constraints of the following problem and its corresponding dual 
problem: Minimize 


T1 — T9 


! A vector in the basis has its z; — c; — 0. 
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subject to 
2131 3-2; — 2 
—z1—2521 
T 2-0 
23 20 


b. Write out the dual to the transportation problem of Sec. 2, Chap. 1. 

6. Construct a two-variable, two-inequality primal problem which has no feasible 
solutions and whose corresponding dual problem also has no feasible solutions. 

7. Farkas" Lemma (Charnes and Cooper [10a], Dantzig [16c], and Hadley [524]). 
Prove the following: A vector b will satisfy Wb 0 for all W satisfying WA — O if 
and only if there exists an X » O such that AX — b. Hint: To prove necessity, 
define appropriate primal and dual problems and apply the duality theorem. 


CHAPTER 6 / THE REVISED 
SIMPLEX METHOD 


1. THE GENERAL FORM OF THE INVERSE 


Upon critically analyzing the simplex computational procedure (see Chaps. 
3 and 4), we see that the essential element which enables us to progress 
from each basie feasible solution to the optimum solution is the explicit 
knowledge of the representation of the vectors not in the current basis in 
terms of the basis vectors. Given this information, we can do the 
following: 


1. Caleulate the 2; — c; elements to determine which vector to introduce 
into the basis or to determine whether the current solution is optimal. 

2. Determine which vector to eliminate from the basis. 

3. Transform the basis and obtain the new solution. 


In Sec. 5 of Chap. 2 we showed that, given a basis B of m-dimensional 
vectors, (P;P» : - - P,), the linear combination that expresses any other 
m-dimensional vector P; in terms of B is determined by computing the 
vector 


X; — BP; (1.1) 
where X; — (xij,rs, . . . 24) is a column vector. We then have 
E; — zyjP1 - 2»;P» 9 E UK: Ms Du 


which is the desired linear eombination. Consider the general linear- 
programming problem of minimizing 
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cX 
subject to 
ÁX — b 
and 
x) 


If we let the matrix B correspond to the first m vectors of A, such that 


BX, u—À b 

X,20 
where X, — (rio220, . . . ,z40), we have a basic feasible solution given by 
X, — B-!b (1.2) 


The linear combinations of all the vectors of À in terms of B can be deter- 
mined by (1.1) for; — 1,2, . . . , n. InSec.1 of Chap. 4 we defined for 
any feasible basis the quantities 


2j; — Cuv 4 CoX5j 4 x EL Es CmXmj (1.3) 
for 21,2, ... ,n, where the c; correspond to the cost coefficients of 
those vectors in the basis. For any 7 we have from (1.1) that (1.3) can 
be rewritten to read 


e coX; zt coB^! P; 3 -m D 2, aroma 


where co — (c,0s, . . . ,c,) isa row vector. Hence, given the m-dimen- 
sional row vector 


"T — coB^! (1.4) 


for a feasible basis B, we can compute the corresponding zj; The vector 
m — (mim... y74) Ós called the pricing or multvplier vector and the 
individual v; are the przcing or simplex multipliers. We price out à vector 
P; not in the basis by computing «P; — c; — 2; — cj. From (1.1), (1.2), 
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and (1.4) we note that the information necessary to proceed from feasible 
solution to feasible solution can be obtained if, for each feasible basis B, 
we have explicit knowledge of B-! and the original data consisting of A, 
b,andc. This fact has led to the development of a computational proce- 
dure to solve the general linear-programming problem termed the revised 
simplex method (Dantzig, Orden, and Wolfe (32] and Dantzig [20, 21]). 

The main difference between the original simplex method and the 
revised procedure is that in the former we transform all the elements of 
the simplex tableau by means of the elimination formulas, while in the 
latter we need transform only the elements of an inverse matrix by means 
of the same formulas. 

Since its development the revised procedure and especially its varia- 
tion which employs the product form of the inverse have been selected for 
use on the larger high-speed computers.! "The two reasons for this are: 


1. For problems whose matrix of coefficients contains a large number of 
zero elements the total amount of computation is reduced? The 
revised procedure always deals with the original coefficients, and 
because the computer codes can be developed to multiply only nonzero 
elements, the total processing time is greatly reduced. Also, the orig- 
inal nonzero elements can be compactly stored in the computer memory. 
The original simplex procedure transforms the zero elements to non- 
zeros as the computation progresses. "The total number of computa- 
tions in the revised method is, in general, less than the number in the 
original method.? 

2. 'lThe amount of new information the computer is required to record 
is, in general, reduced, since in the revised procedure we need to record 
only the inverse and the solution vector, while in the original method 
the complete simplex tableau has to be recorded. The recording is 
even further reduced if the product form of the inverse is used. 


Before describing the systematic computational rules for the revised 
procedure, we wish to show how the inverse of each new basis can be 
obtained from the preceding basis by applieation of the elimination formu- 
las. Here we are given an old basis B — (B,P - - - Pi - - - P,), which 
differs from the new one in that vector P, has replaced vector P,. Let 


! See Dantzig and Orchard-Hays [29] and Sec. 2 of this chapter. 

? Problems such as the caterer problem, production-scheduling problem, and inter- 
industry problem described in Chap. 11 and many other problems are of this nature. 

* Wagner [99] gives a complete comparison of the original and revised simplex 


methods. "There he shows that, for n » 3m, the revised method is better in terms of 
the number of computations. 
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B — (BP; - - - P,-: - - P,) represent the new basis. We then have 
B-3B — BOS(DBiB.--- PB -- DB, 
1 0 0 
UERCNG 0 0 
RT "ACONLOREU E UEI (1.5) 
0-0 0 1 
and from (1.1) 
B-—B- B-!(BiP; SEDES 
1 0 Xi 0 
1 X2 0 
Ex Qc cts epe tete (1.5a) 
0 0 es s py Sut on 1 


If we let bi; equal the element in the :th row and jth column of B-! and 
let bi; be the corresponding element of B-!, we have the 5;; given by the 
elimination formulas 


bi -— bi — 2 Xik for 2 z ! 
lk 
(1.6) 
Bye 
| use Rec 
Xu 


The validity of this transformation can be verified by direct multipli- 
cation of (BB) to determine whether the product yields the identity 
matrix For example, we have the inner produoct of the first row of B-! 
and the first column of B given by? 


b bis. 
(v. -- 2 au) 033 F c i au) üspud- sot (o- Ils 2 G1 
X1 X X 


(b11231 s Disuspel oe sls b15051) 
e (bi1a31 4-510931 d- : : : c 51,051) 


or 


Tik 
m (n 
! From (1.5a), B-! in terms of B7! is given by B^!B — I; whereliisthe right-hand 
side of (1.5a). Then B — BI, B-! — (BIj)-!, and finally, B^! — I;!B^'. In Sec. 2, 
we show that I;! — E!', where E' is an identity matrix except for the ith column. 
? This product is obtained by the rule of matrix multiplication described in Sec. 1 
of Chap. 2. 
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The first term of (1.7) is the inner product of the first row of B^! and P1, 
which from (1.5) is equal to 1; and the second term of (1.7) contains the 
inner product of the /th row of B-! and P, which equals 0. Hence (1.7) 
is equal to 1. In a similar manner we can show that (B-:B) - I, and 
hence formulas (1.6) do generate B-!.T 

We next describe the computational format for the revised simplex 
method as developed by Dantzig [19] and Orchard-Hays [81]. We shall 
not attempt to develop the revised method fully, in the way the original 
simplex method was presented in Chaps. 3 and 4, but the reader will be 
able to determine the relationship between these methods. 

In order to facilitate the use of the revised procedure, we shall intro- 
duce and define below two new variables z4,,41 and z545.2. 

For the revised procedure we write the general linear-programming 
problem as follows: Maximize 


Xn-m41 (1.8) 
subject to 

diti dau. po E ON — bi 
(1.9) 

GmiX1 T Oo 9372 3E it us JE OmnXn s Ds 

Quid uult c S SER 

X1 pz 0 

X2 P. 0 
(1.10) 

Xn z0 


Here we assume all b; » 0. Since 
nme gcc eC 10 oT" cM (E 


and since the linear-programming problem that minimizes an objective 
funetion 1s basically the same as the one which maximizes the negative of 
this objective function subject to the same constraints, the above problem 
is equivalent to the problem of minimizing 


C111 1 C212 4T "ais 4j CnXn (1.8a) 


T The matrices formed by vectors P;, P,, and P, in the example of Sec. 2, Chap. 4, 
illustrate this transformation. 
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subject to 

1:31 lo eda 5 t S pd. s bi 

trucs cce TUE (1.9a) 

G1 AF Gm 3X2 dr VERD 4E Gmnin — bs 

X1 zm 0 
Xo p 0 
(1.10a) 
la 0 


Since the objective function in the revised problem consists of only 
E5441, that is, the value of z;4,,411s the negative value of the minimizing 
objective function, we do not restrict the sign of z,,,,,1in (1.8) and (1.9). 

As in the original method, the revised procedure has to begin with a 
basis consisting of an identity matrix whose unit vectors correspond to 
either real or artificial vectors. If the revised procedure starts with 
artificial vectors, we again have the problem of determining a first basic 
feasible solution (if one exists) and then continuing on to the optimum 
solution. We shall let Phase I refer to the part of the computation that 
determines feasibility and Phase II to the part that develops the optimum 
solution. In order to generalize the concept of artificial vectors and to 
facilitate the computation of Phase I, we calculate a *'redundant'' equation 
defined as 


Q2, X1 x Qm 2, 2X2 "b oe) LE T Om--2, nn 4 Xm-n42 — boo 
where 
m 
Üs4nj 7 — D, 0d gea. Qn 
diam 
" (1.11) 
boo v Y b; 


We see that 2,4»,; is just the negative sum of the coefficients in the jth 
column of the coefficient matrix A and that b,,,»1s the negative sum of all 
the b; The significance of the 'redundant" variable z,4,,4,» will be 
described below. If we let c; — aà541, j, we then rewrite the revised prob- 
lem (1.8) to (1.10) to maximize 


(1.12) 


Tn--m41 


subject to 


[Chap. 6 
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where the nonnegative artificial variables z,,;for? — 1,2, . . . , m have 
been added to the corresponding equations. "The artificial variables ean 
be interpreted as the measure of the error between the sides of the original 


equations of (1.9) evaluated for a vector X — (xis, . . . ,2,) which does 
not satisfy all the constraints of (1.9). From (1.11) and (1.13) we have 
X1 zi X242 - ee ri "E X n4-m -F Xn-4m-42 — 0 


-The variable z,,,4» can then be thought of as representing the negative 
of the absolute sum of errors of an approximate nonnegative solution to 
the equations. Since z,4,,27 0 for — 1, 2, .. . , m, it is clear that 
Z&4m42» cannot be positive. 

As written in (1.12) to (1.14), the revised problem consists of m 4- 2 
equations in n 4- m 4- 2 variables. Here a basic feasible solution will 


gonismemoL varsables.from the set (2:95, «c E nd minu): 
The signs of the last two variables are unrestricted, and they will always 
bein the solution. The variables from the set (xi, . . . ,z,) that are 


in the optimum solution represent a corresponding basic maximum feasible 
solution to the problem of (1.8) to (1.10) with z,,,41 the value of the 
objectivefunetion. "Thisoptimal solution is also a basic minimum feasible 
solution to (1.8a) to (1.102), where 


—Tn-m4i 7 Cui x LP E CnXn 


is the corresponding value of the objective function. The computational 
procedure for the revised problem starts out with the m artificial variables 
É&b ooa &nd variables 445541 8nd z4444» in the solution. We 
must employ the Phase I procedure to find a first basic feasible solution. 

In Phase I we first consider solving the problem of maximizing 2,542, 
subject to (1.13) and (1.14), with both z,4,441 and z,,,44» unrestricted as 
to sign. If the maximum value of z;,,45» 1s zero, then all the x44; for 
$21,2,...,m must also equal zero, and the x; for? — 1,2, .. . ,n 
of this "preliminary maximum solution" represent a basic feasible solu- 
tion to the problems (1.9), (1.10) and (1.9a), (1.102). Ifthe maximum of 
Zn4m42 18 less than zero, then this implies that at least one artificial variable 
is in the Phase I solution with a nonzero value and hence that no feasible 
solutions exist fortheoriginal problem. Intheformercase wethen goon to 
PhaseII, where we maximize z, 4,41, subject to (1.13) and (1.14), while keep- 
ing z4454» equal to zero. It should be noted that the set of basic feasible 
solutions found in Phase II might contain artificial variables with zero 
values. The final Phase II solution is the desired optimum solution. 

We shall next describe and illustrate the complete computational pro- 
cedure for the revised problem which starts with a fullartifcialbasis. In 
the computational tableau for the procedure we need to keep record of 
only the following information: the variables in the solution, the corre- 


— 
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sponding values of the variables, and the inverse of the c basis. In 


developing new solutions, -we-müst have a. means of caleulating the 2j 
-ierms. We shall do this by having the vector defined by (1.4) available 


for the current basis. We can then determine which vector is to be intro- 
duced into the basis and caleulate (1.1), where P; corresponds to the vec- 
tor being introduced. We next find out which vector is to be eliminated, 
using the method described in Chap. 4, and then transform the inverse of 
the current basis by the elimination formulas to obtain the inverse of 
the new basis. 

Let us detach the required coefficients from (1.13) and arrange them 
in an m 4- 2 X n matrix as follows: 


Q11 012 Q3k Q1n 
[P3 022 Qok 25 

À - [an 12 Q1 ln 
[771 Q2 Gmk Gmn 
Gut. Umw»po t" * mig 0*0 ma3.m 
Qm-42, T Qn--2, 2 ddr Q,4-2, k als Qn 4-2, n 


We shall denote each column vector of À by Á;. Let us compare the 
information in matrix À with the initial artificial-basis computational 
tableau for the original procedure for the minimization problem described 
in Chap. 4 (see Tableau 4.4). We note that the matrix formed by rows 
1,2, ..,m,m-l,0-PF2.aund the column& Py, P5, . . .. Pl. ottbe 
tableau is equal to matrix À, except that rows m 4- 1 and m -- 2 of À are 
the negative of the corresponding rows of the tableau. Asin the original 
method, row m 4- 2 of À will be used in computing the z; — c; elements 
when artificial variables are still in the solution, and row m -- 1 will be 
used when they have been eliminated.! 

Since our starting basis for the revised procedure consists of an iden- 
tity matrix, its inverse 1s also an identity matrix, and this information is 
recorded in the following m -- 2 X m 4- 2 matrix: 


Quee coss a nudn 
pope ELE 
(39.5 sesrvüsce sido Oe 


! Dantzig and Orchard-Hays [29] write the revised problem with the equations 
containing variables 75,541 and 44,42 as the zeroth and first equations, respectively. 
They renumber the equations to make z,,441 — zo and QUndemi o EESEgS 
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The first ?» rows and m columns of U represent the inverse of the starting 
basis B. The last two rows of U will be used to determine which vector 
should be introduced into the basis, with row m 4- 2 generating the infor- 
mation in Phase I and row m -4- 1in Phase II. We will embed the matrix 
U in the starting computational tableau and transform its elements as 
deseribed below. Let us designate the elements of U by wu; and its rows 
by the row vectors U; — (uu,Ui, . . . ,Ui,m42). For convenience in 
describing the computational procedure, we shall let U; represent both the 
original and transformed versions of the ith row of U. The matrix U 
will be the inverse of an m -- 2 X m 4- 2 matrix whose columns are 
vectors of (1.13). 

We arrange the matrix U and the initial solution of z,,; — b; for 
$*-1,2,...,m;Zap41** 0; and $4154» € 0544 9s Shown in the. start- 
ing tableau of Tableau 6.1. "The computational steps for the phases are 
as follows: 


Phase I. Artificial variables in the solution with positive values 
STEP]. Ifz,j4,,: « 0, compute 
à; — Us 5À;f 


— Um-2, 103j am Um-4-2, 202j Ei 
SV T m2, ma20m42, j dis do neto dE 

and continue to Step 2. 

If x4444» — 0, go to Step 1 of Phase II. 

STEP 2. If all à; 2 0, then z,,54» 1s at its maximum, and hence no 
feasible solution exists for the problem (1.8) to (1.10). 

If at least one à; « 0, then the variable to be introduced into the 
solution, z,, corresponds to 


Ln ze min y 


If more than one of the 5; are equal to the minimum, select 8, such that 
kis the smallest index. "This is an arbitrary, but computationally sound, 
rule for breaking ties. 


t In Phase I for the revised problem the objective function to be maximized is 
Zm4ni2. ELHence, all cost coefficients except C445,» — 1 for this maximizing problem 
are equal to zero. Similarly, in Phase II the objective function to be maximized is 
Zm4n41, ànd the cost coefficients are all equal to zero except €&444: — 1l. The quanti- 
ties 8; of Phase I and y; defined in Phase II can be interpreted in two ways. "They 
are equal to, respectively, the artificial and real 2; — c; elements of the original sim- 
plex procedure adapted to maximizing the objective functions of the revised problem. 
'The criterion then states that the optimal solution has been reached when all 2; — 
c; 2 0. For the corresponding iteration, the quantities 2; and y; are also equivalent 
to the c; — z; elements that would be computed in solving the minimization problem 
defined by (1.8a) to (1.10a). Here the criterion states that the optimal solution has 
been reached whenalle; — 2; 7» 0. Insetting up the revised problem, we shall always 
write the original problem as one to be minimized and then determine matrix À, where 
&m41, j 7 C; of the minimization problem. 
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sTEP 3. Compute 


rg — UiÁA. 
— Wiüik d- Wisüak d- c coc cb Wi mae m2, k 
fori21,2,...,m,m J- 1, m 4-2. The variable z; to be eliminated 


from the solution corresponds to the ratio 
0, — min — — — pues dio oe atom 


where the ratio is formed only for those z4 7 0. If there are ties for the 
minimum, we select | to be the smallest index.! [For discussion purposes 
we have taken the ratios for the values of index? — 1,2, . . . ,m. The 
ratios z;o/r; are actually taken for those 7 whose corresponding z;o are in 
the basic solution to (1.13).] 

sTEP 4. "The new values of the variables in the basic solution are 
obtained by the formulas 


X10 : 
gu coge qu fori; Ek 
Xu 
UNE T0 
T m 
: Tux 


The new elements of the matrix U are transformed by 


/ ui 


42 9 d; o due fori zl 
E Tk 
Ul. 
jm J 
uy -m—— 
Tk 


Under this transformation the (m -4- 1)st and (m 4- 2)nd columns of U 
wil never change. "These unit vectors enable us to add the correct cj 
when we compute the c; — zj, Using the transformed rows of U as 
shown in the transformed section of Tableau 6.1, the steps of Phase 1 
are repeated until it is determined either that no feasible solutions exist 
or that the value of z4,,,,» — 0. In the latter case we go on to Phase II. 


Phase II. No artificial variables in the solution with positive values 
STEP l. Here z44445 — 0. Compute 
y; 7 UnjiÁ; 
— Umi, 1015 TF Uma, 202; 47 
NEST 4 Vm--1, m4-20m--2, j j- lL- rus Hace 


1 In this situation the resultant basic feasible solution will be degenerate. "We 
shall discuss in Chap. 7 thetheoretical computational rule for the degenerate case given 
by Dantzig [19]. It might be noted that the originators of the revised procedure do 


not incorporate the *degeneracy routine" in the computer codes developed at their 
research center (Orechard-Hays [82]). 
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STEP 2. If ally; — 0, then z,,,441 is at its maximum value and the 
corresponding basic feasible solution is an optimum solution. "The nega- 
tive value of z,,,4,411s the true value of the objective function that was to 
be minimized. 

If at least one y; « 0, let 


yi; — min y; 


Variable z, is selected to be introduced into the solution. Ties in select- 
ing y, are broken by selecting the variable with the smallest index. 
sTEP 39. Compute 


Xa -— UA. 
s« Woiy p Ueürpcb toto ona a0m2; E 
fori1,2,...,m,m--1,m--2. The variable zi to be eliminated 


from the solution is determined by finding the ratio 
(, — min — — DK p e p 


for those zi » 0. (See Phase I, Step 3, for actual range of index 7.) 
'Ties are broken in the usual fashion. If all z4 € 0, then the procedure 
has yielded a solution whose value of the objective function can be made 
arbitrarily large. 

sTEP4. "Thenew values of the variables are obtained by the formulas 


X10 ; 
Q4 7 fo — —4i for? z k 
Vk 
27 X0 
(p iai. 
4 Vu 


'The elements of the matrix U are transformed by 


/ ug 


Mc E IE fori zl 
Tu 
Ui 
P —. J 
MILLS 
Tk 


The steps are repeated until an optimum solution with either a finite 
or an infinite value of the objective function is determined. 

The initial solution to (1.12) to (1.14) and the above steps can be 
arranged and carried out by the computational scheme shown in Tableau 
6.1. A column has been set aside for the recording of the z;; associated 
with each inverse. 'Theinverse matrix U consists of the m 4- 2 X m 4- 2 
array enclosed in heavy lines. Thelast two columns of the transformed U 
donot change. "The inverse of the m-dimensional basis selected from the 
P; of (1.13) without the (m 4- 1)st and (m -r 2)nd rows is enclosed in 
heavy and dashed lines. 
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Tableau 6.1 


Matrix U 


I f basis Hei U; Ak 
nverse of basis, B 
Index of : (B-!PX) 


variables Value of 
in variables s 
solution UsiAk 
Um4:AÀk 


Row 
index of 
tableau 


Inm 


Ine4mal — 


Inome2 


Mc 
] 
uma 'O 


Umen d üume1s77* EROS 2550 e ET 


, , 
Um4«2,1 U'm42,2 7 * * tt u'ma2,m | O 


If the revised problem has unit vectors in its explicit mathematical 
statement, these vectors can be used in the starting basis for the revised 
procedure if their eost coefficients are equal to zero; for example, slack 


vectors. If these vectors are used, formulas (1.11) must be changed to 
read, respectively, 


(nem » Qi d e 
izB 
and 
bas — — Y bi (1.16) 
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where 7 »« B represents the set of indices of the rows that still require an 
artifieial vector. If m appropriate natural unit vectors are available, 
then b,,» — 0 and the procedure starts with Phase II. 

To illustrate the revised procedure and to help point out the relation- 
ships between the two simplex methods, we shall next solve the example 
of Sec. 3, Chap. 4, using the revised procedure. 


Example. Maximize 


X1 22$ 4- 925 — x4 


subject to 
$i d- 2x5 4- 323 3 
2xi-d- zx. 52, — 20 
q1-- 24» -- zs -- 24 - 10 
aud v; — 0. 


Here m — 3 and n — 4. "The objective function for the corre- 
sponding minimization problem 1s —z; — 2zs — 323 4- z,. 

For the revised procedure with a full artificial basis we rewrite 
the example to read: Maximize 


Ts 
subject to 
Z1 4- 223 4- 323 RISE — 15 
2zxi-- z.24- 523 xs -—21220 
€3-- 2x32 -d- Xa v4 oy — 10 
—41 — 223 — 3zsa -- 24 uu -—-0 
—4z; — 5x» — O3 — x4 pop cm 40 


The coefficients of the first four variables for the fourth row are 
equal to the corresponding c; as written in the original objective 
function to be minimized. The coefficients of variables zi, 2», 23, 24 
and the right side of the fifth equation were obtained by formulas 
(DPI: 

The À and U matrices for the example are 


» 
! 
A 
t2 
- 
-— 
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and 
1^ 520:00 70:9 
0g. «20-9 
U-T90 0 I 0 QU 
0:590. 1.0 
0-0 0 0.1 
The starting tableau and the sequence of iterations are shown in 
Tableau 60.2. 


2. THE PRODUCT FORM OF THE INVERSE 


In Sec. 1 we developed the computational formulas (1.6) that enabled us 
to compute the inverse of a basis B which differs by one vector from a 
basis B with known inverse B^!. Making use of the notation of Sec. 1, 
we have 


B-e(DbP 4B vB) 
B-2(BIP:--PRo-- P.) 
bu b bu bis. 
Biz bu Un bu bis 
Da loyer: b, Dein 
and 


Xank 
Also 
bs bis bu [TER 
B-: E b bis bu bs 
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Tableau 6.2 


Index of 
variables 


Row 
index of 
in 


tableau . variables 
solution 


Matrix U Zik 


Phase I 


5 
6 à; — Un4:Às — —9 
z 20 
zEESUEM vl 
8 X53 5 
9 
1 —3$6 O0 "OL 
0 M 0 i 0 0 —- $0) Unm42À2 c — 164 
DN MICE M zs — 15 
0 2$ 0 1 0 zi 7 
0 96 0 0T 


3. Third Iteration 


ó4 — Us42À4 ^ —1 
z; 15 


I7]4 7 


z$ — 0 

RUE q3rE LU EpÁY — 0-94 
" Z4 209 

ws X41 7 2 

All y; 20 


Optimum solution: 

zi x2 — X3 — 5$,14 — 0 

Value of objective 
function: 
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where 


(y ckdy — EB, dordsed 


X 


px 


sm 
d Tk 
Let us form the m X m matrix 


Zik 


1 0 T 0 
XE 

0 1 cs 0 
XI 

m 

0 0 t 0 
Xlk 

0 0 duh 1 
XI 


The matrix E! is called an elementary matrix, 1.e., an identity matrix 
except for the ith column. "The reader can readily verify by direct 
multiplication that the matrix product 


E!B-! 2 B-: 
Hence the premultiplication of B^! by E! is equivalent to the applieation 
of formulas (1.6). Asin (1.6), this is accomplished with the knowledge of 
only X, and B^. If we let 


Mp amo esi i for? sé (2: D) 


Mi me ccs (2.2) 


then E! can be written as 


l yu 0 
B. 0 yu 0 
0 uml il 


In fact, to write out the explicit representation of the elementary matrix, 
we need to know only the column index ! and the set of m elements ya. 
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Using a shorthand notation, we can let 


B yi e esit es Sw) 


In the original simplex method we usually initiate the computation 
with either an artificial or a real basis Bo which consists of an m-dimen- 
sional unit matrix. "The inverse of B, is, of course, an identity matrix; 
let us denote Bg! — I - E, The inverse of the next basis Bi, which 
differs from B, in that vector P, has replaced vector P;, can be computed 
by multiplying 


E/E, —- Br! 


where E;' is the elementary matrix obtained in making the first change of 
basis. In general, given the associated set of elementary matrices, the 
inverse of the pth basis can be obtained from 


EJEí4 2 EJSEQ B. 


We should note that, in each E,/!, the index | corresponds to the positional 
index of the column in the preceding basis that is being replaced. We 
see that, for any basis B,, given the corresponding set of elementary 
matrices, we can obtain B,^!, and hence the methodology of the revised 
simplex procedure should apply. 

In the product form of the revised simplex method the computation 
begins with the m 4- 2 X m -- 2 identity matrix U. U is interpreted as 
being the inverse of the initial basis to the problem stated by (1.12) to 
(1.14). 'The associated elementary matrices are of order m 4- 2, and 
the rules of Phase I and Phase II of Sec. 1 apply, with the following 
modifications: Since, in the general form of the revised method, we always 
have the explicit knowledge of the "inverse matrix" U, we can immedi- 
ately calculate in Phase I, Step 1, the necessary à, — U,,;À; In the 
product form, however, we have to determine first the current U,,,5, 
and this is best done by computing 


NSGEJE 4 EJ 9D, us 
where 
IN oso 100 7.0.1 


is à unit row vector of dimension m -4- 2. "The caleulation can be effi- 
ciently carried out by interpreting the product to be 


([(Vo,4E,)E,A] - : : Ei] 
We should note that, in the product of any row vector 


Moss rs ut osa) 
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and an elementary matrix E,', that is, in the product W — VE,!, the 
elements can be computed by the formulas 


W; i fore sé i 


jos » Uii 


1 


In Step 3, the vector X, is best computed by 


X, (Eg[Ej-4 - - * (EJÀgI] 
In computing these products, we note that, in the product of a column 
vector À; — (aij, . . . ,0445,;) and an elementary matrix E;!, that is, in 


E,'A; — D; the elements can be computed by the formulas 
di — ai 4- yaaiy for? 25 i 
dij — yudi 


The new elementary matrix and the current solution are developed in 
Step 4 by formulas (2.1) and (2.2), and 


, 


X;y 7 4o d- alo foris k 
/ 
Xko — QyuXio 


respectively. 
The above modifications apply to Phase II, except that in Step 1 we 
compute 


(Gu EZ)E, a] : : ^ Ej] 2 Usa 
where 


Vau ix (0,0, "A s De 050) 


is a unit row vector of dimension m -- 2. 

The major advantage in generating the inverse for each basis by 
means of elementary matrices 1s that only a minimum amount of informa- 
tion, namely, E,', need be recorded. "This is extremely important when 
one must use a computer which has alimited high-speed storage. Here we 
need only record in the shorthand notation that E,! — (lys, . . . jysas) p 
instead of recording the full inverse U after eachiteration. For the more 
recent computer codes, the revised simplex procedure, using à product 
form of the inverse, has been employed. 


REMARKS 


The material in this chapter is from Orchard-Hays [81] and Dantzig and Orchard- 
Hays [29]. For additional remarks on the computational elements of the general 
form of the revised procedure, the reader is referred to Dantzig [21]; for the product 
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form of the inverse, see Dantzig, Orchard-Hays, and Waters [30], and, in genera], 
Orchard-Hays [83a]. 


EXERCISES 


1. Solve the example of Sec. 1 by means of the product form of the inverse. 
2. Solve the following problems by both forms of the revised procedure: 
a. Maximize 


5zi m do 4 AU gum 10z, 4 T Cs 


subject to 
dri — X» — X3 4 
SE cw ub uS Duk ec ER -1 
2zi d- 2» 4- 223 ToXxs-—' 
and z; 20 


b. Minimize 


— 21-1 222» 
subject to 

5r; — 2z9 X 3 
zi-d- z321 
—3zi-- zs € 8 
—8z; — 8:3 € 2 
2i 22x) 
2728720) 


3. Solve Exercise 2b by graphical methods. 
4. Solve the following problem by the original simplex method and by both forms 
of the revised procedure: Minimize 


Gee Hp ER 
subject to 
«m —— — 2x6 — 5 
X2 2x. — 325 d 2&4 — 38 
Z3 4- 2z4, — 5xs 4-605 — 5 


and 
T; E 0 


CHAPTER 7 / DEGENERACY PROCEDURES 


A degenerate basic feasible solution to a linear-programming problem is 
one in which some z;o, where the index ? corresponds to a vector of the 
admissible basis, is equal to zero. That is, if the vectors (P;P» * * * P) 
form an admissible basis, then, in the degenerate case, the corresponding 
nonnegative linear combination 


Zi Pi- daoPg ort 4 Sob m P, 


has at least one z;; — 0. In Chap. 4 we assumed in proving the validity 
of the simplex procedure that all basic feasible solutions were non- 
degenerate. "This assumption was necessary in order to demonstrate 
that, for each successive admissible basis, the associated value of the 
objective function is smaller than those that precede it. Hence, we will 
reach the minimum solution in a finite number of solutions, since there 
are only a finite number of possible bases. "This proof breaks down if we 
admit the existence of degenerate basic feasible solutions. "The latter is, 
of course, the more realistic situation. For a degenerate solution, we 
have the possibility of computing a 6, — zio/zi, [see Eqs. (1.6) and (1.7) 
of Chap. 4] for which 69 — 0. "This choice of | vector. P; to be eliminated 
and P, to be introduced into. ihe basis will give a new feasible solution 
whose value of the objective function is equal to the preceding one.! It 
is then theoretically possible to select a sequence of admissible báses s that 
cycles, i.e., a sequence of bases that is repeatedly selected without ever 
satisfying the optimality criteria and hence never reaches a minimum 
solution. "The possibility of cycling is crucial only if the current basic 


! Here weshould distinguish between a multiple solution and a degenerate solution. 
A multiple solution has the same value of the objective function as the preceding 
solution because z, — cy — 0 with P, not in the basis and 6, » 0, while a degenerate 


solution will yield à new solution with the same value of the objective function 
because 09 — 0. 
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feasible solution has more than one r;; — O0 (see Exercise 2, Chap. 4). 
With at least two z;; — 0, it could be possible to have a tie with 0) — 0 
in the selection of the vector to be eliminated from the basis. It is also 
possible to have a tie in computing 6, even if the current solution is not 
degenerate. Here, of course, 09 7 0, and the new solution will have an 
improved value for the objective funetion. However, because of this 
tie in the old solution, the new solution will be degenerate. 

From the above discussion, we see that any device developed to over- 
come the degeneracy restriction must be concerned with the determina- 
tion of a unique 6, and hence of the index | of the vector to be eliminated. 
À number of such techniques have been developed (Dantzig [17], Charnes 
[9], Wolfe [103], and Dantzig, Orden, and Wolfe (32]). Computational 
experience on digital computers has minimized the importance of these 
techniques, since there have not been any practical problems that have 
been known to cycle. In other words, the successful solution of hundreds 
of problems has not hinged on the development of these techniques. 
For this reason, these procedures have not been incorporated in most 
computer codes. What is important, however, is that these devices 
"make the simplex method available, without blemish, as a crisp tool for 
proving pure theorems" (Hoffman [59]). 


1. PERTURBATION TECHNIQUES 


Geometrically, a degenerate situation 1s one in which the vector P, 
lies on à bounding hyperplane or edge of the convex cone determined by 
its basis vectors. For example, in Fig. 7.1 vector P, can be expressed 
as a positive combination of P; and P; but as a nonnegative combination 
of P,, P, and P; with z;o — 0. However, if we move, i.e., perturb, the 
vector P, in such a manner that it lies inside the convex cone determined 
by vectors P;, P», and P;, then the corresponding solution will be non- 
degenerate. We can do this by taking a positive linear combination of 


FIGURE 7.1 
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these vectors and adding it to Py. However, we wish to do this so as not 
to destroy or hide the original problem. Hence we take our positive 
combination small, and, as suggested by Charnes [9], we would let the 
positive combination for our example be 

eP, 4- &?P, -- eP; 
where e is some small positive number. — The constraints of the new prob- 
lem would be 

ZieP Z39P» 4- S39Ps — P, 4- eP, 4 €&P, 4 P; — P(e) 


Its geometric interpretation is given in Fig. 7.2. 


FIGURE 7.2 


For the general linear-programming problem we wish to perturb the 
problem in à similar manner so as to ensure that, for any possible admis- 
sible basis, the corresponding solution will be nondegenerate. (Hence 
the perturbation technique and other schemes are procedures for justify- 
ing the nondegeneracy assumption of the simplex method.) "To accom- 
plish this for the general problem, we rewrite the constraints 


"b Sil T z9P» EE MO ES 3m WB ais e Na SS LA rl m Po qe m 
to read 

z1Pi 3r x)Ps; 4d Ore e ES diktn 3 DEEP E dub -— P, ss eP; 4r €P; 

J- ded l e"P,, 4 "f J- e P, L— P,(e) (1:2) 

Let us assume that the vectors Pj, P», . . . , Pj, form an admissible basis 
B. "Then a solutionto (1.1)is 

Xvo-BcP-0 (1.3) 
and a solution to (1.2) is 

Xo(c) E B-!P(e) (1.4) 
Let 


Hoc (1.5) 
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Then (1.4) 1s given by 
Xe) — B-!P, -- ceB-!P, 4- e B-!P, 4- 


s 4- e"B-1P,, -4- TE 4 e&B-!P, 
exe pex i usuPE spe Ma 
We note that, since B consists of vectors Pj, P, . . . , P,, X; - B-!P; 


is an identity vector with the unit element in position j for j — 1, 2, 
.,m. Hence the zj;;(ce) are given by 


Sue) — di Dn ers (1.7) 
j-71 
Or 
fi(e) — £io d- é 4 »» ei; (1.8) 
jmd 


From (1.8), and by taking e » 0 but sufficiently small, we can make all 
Zi(e€) » 0fori — 1,2, .. . , m. (Wecan always rearrange the problem 
to make the basis vectors the first m.) In determining the vector P; to 
be eliminated from the basis for the problem of (1.2), the criterion that 
ensures the feasibility of the new solution is the selection of 


0, — —- min 2" — min 20 (1.9) 
Xu D Tik D Tk 
for zu » 0. We see from (1.8) that ties cannot occur since ijo(e) is the 
only variable that contains e. 

Once a solution to the linear-programming problem with the con- 
straints (1.2) has been obtained, by letting e equal zero, we shall have the 
corresponding extreme-point solution to (1.1). Inactualpractice, it is not 
necessary to select an e and rewrite the problem as was done in (1.2). 
From (1.6) and (1.9) we note that the information required to determine 
05 is the zio of problem (1.1) and the coefficients ofthe &. — Thisinformation 
is all contained in the simplex tableau of the unperturbed problem. For 
e small enough, we note that the significant coefficients of e are the first 

Jones, starüng with ? — ly. eh.  lhe procedure can then-be rou- 
tinized as follows: If, for all zio in the basie solution, the ratios Z;o/z;. for 
Zi; » O0 yield a unique 0, — min (Zio/zi), then vector P; is uniquely deter- 


mined. If, however, there.are ties for the minimum. for.some. set. of 
indices, we compute starting with ; — 1 the ratios.z4/za for all rows ? 
dnt this set and compare t these ratios. The index that corresponds to the 
algebraically smallest ratio determines the index of the vector to be 
eliminated. If there are still ties in computing the minimum, we form 
the next ratios for the tied set of indices for column j -- 1 and repeat the 
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analysis. For example, if for the admissible basis (BiP» * - P,) we 
have 
X10 —— X90 
[7] LL ——— L—— ——- 
Xik X 2k 


we then compute z1;/zi. and 221/25, and compare. If 
mi e — ior? 1,2 


then vector P; is eliminated. If 


then P; is eliminated. In both cases P, is the vector introduced. (The 
selection of the new vector does not depend in any way on the degenerate 
situation.) If zi/zi — zn/zs, we form the ratios zis/zi& and zss/Z5& 
and repeat the comparison until the tie situation is broken. We know 
from (1.8) that this will happen for some j. "The simplex tableau is 
transformed in the usual manner with the selected rj; as pivot element. 
The new X;(c) solution to (1.2) will be nondegenerate for e sufficiently 
small, and the above process is repeated until à minimum solution is 
found. 

The above perturbation scheme is ideally adapted to the original sim- 
plex method since all the necessary information to execute the scheme is 
readily available. For the revised procedure a different approach to the 
problem also yields an efficient technique that requires knowledge of the 
inverse of the current basis (Dantzig, Orden, and Wolfe [32]. . We shall 
not discuss the development of the method except to describe the rule 
to be used when breaking ties in the computation of 6,. "This rule is 


given in Dantzig [21] and is: "If two or more indices l;, l, . . . are tied 
for the minimum, divide the corresponding entries in the first column of 
the inverse by zi, x, Xi, x, . . . , respectively, and take the index of the row 


with the minimizing ratio for |. If there still remain ties (for the mini- 
mum), repeat for those indices that are still tied, using as ratios the cor- 
responding entries in the second columm of the inverse divided by their 
respective z;,,. Hatios formed from successive columns of the inverse 
are used until all ties are resolved. (Since no two columns of an inverse 
can be proportional, a unique | will be chosen by the last column.)" 


2. EXAMPLE OF CYCLING 


The literature contains very few examples of problems that cycle when 
they are solved by the original simplex method (Hoffman [58] and Beale 
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Tableau 7.1 


II. 


III. 


IV. 


VI. 


VII. 
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Tableau 7.2 


—4 zm 
—250 | —1094 


2 


VI. 


[4). "The problem of Beale [4] is diseussed in terms of its dual. We shall 
next ilustrate the cycling phenomenon in terms of the primal by a 
previously unpublished example due to Beale. 

'The problem is to minimize 


— 342i -- 150z; — Ll$or; J- 0x, 
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subject to 
Ll4zi — 00r; — légr: d- 9z4 4- m5 -—-0 
ljxi — 90r; — Légzs JF 3x4 d ze -0 
Xs "Pd -—1 
and 
2 0 


In showing how this problem cycles, we shall select for the vector to 
be eliminated the one whose row index is the smallest. (This is the rule 
used in Chap. 4 for breaking ties.) In computing 6, ties arose in the 
first, third, and fifth solutions. 'The steps in the process are shown 
in Tableau 7.1, in which it will be noted that the seventh solution is 
identical with the first solution. If we should continue the process with 
the seventh tableau, we should only repeat the previous solutions and 
never reach the minimum solution. 

However, by applying the degeneracy procedure described in Sec. 1, 
we select a different sequence of solutions and do determine the minimum 
solution. The steps in this computation are shown in Tableau 7.2, 
where the sixth solution gives the final answer. "There is no recurrence 
of any solution. "Ties occur in computing 6, in the first and third solu- 
tions. "The first three solutions in both Tableaus 7.1 and 7.2 are the same. 
'The main difference occurs in going from the third to the fourth solution, 
where in the second case we eliminated P; instead of Pj. The minimum 
value of the objective function is — 159. 


EXERCISES 


1. Show, using the trigonometric functions, that the following problem (Hoffman 
[58]) will cycle after ten iterations if the standard rules of the simplex method are 
applied. Note that the cycle is started by introducing P,followed by P;. Determine 
an appropriate range of » and a value of 9. 

Minimize 

co8ó6—]1 


UU" Z4 d wzs 4d- 2wz, 4- 4 sin? à za 4- w(2 — 4 eos? $)z, 4- 4 sin? $ zio 


4j w(1 — 2 cos $)xXu 


subject to 
X1 — it 

Z5 - cos $ó 14 — w cos $ z$ 4- cos 29 zs — 2w cos? $ xz; -- cos 26 zs 
-F 2w cos? $ zg 4- cos à zio d- w cos ó zii 


sin? $ 
Pow M 


0 


ll 


tan $ sin $ 


Puce c Epp eos quer rs d-cos20 1;— 2 


tan $ sin dre 


tan $ sin 26 
w 


-F eos 24 z, — 190 d- eo8 $ z11 — 0 


2. Solve the problem of Exercise 1 by using Charnes' rule for resolving ties. 
3. Solve the cycling example of Sec. 2 using the revised simplex method and the 


corresponding rule for breaking ties. 
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4. In Exercise 1, let the intersection of the second and third rows and the fourth 
and fifth columns be denoted by the 2 X 2 matrix A. Notethat A? is the correspond- 
ing 2 X 2 matrix associated with the sixth and seventh columns, A? with the eighth 
and ninth columns, A* with the tenth and eleventh columns, and A* — I with the 
first and second columns. Attempt to construct a new linear-programming problem 
with a similar set of matrices of finite order that will cycle. 

b. Starting with vectors P;, P», P; as the initial basis, will the following problem 
cycle when the regular simplex rules are applied (Yudin and Gol'shtein [106]): 


Maximize 
3 p gu ce 
subject to 
T1 -F 2x34 — 3x4 — 5z5 4- 0x; — 
Z3 4- 6x; — 5x, — 3r; 4d- 2x5 L0 
Jar; 4 z4 2r; d- 46 2-1; 1 
mox) 


6. Prove that the minimum length of a cycle in a linear-programming problem is 
six iterations (Yudin and Gol'shtein [106]). 


CHAPTER 8 / PARAMETRIC 
LINEAR PROGRAMMING 


1. THE PARAMETRIC OBJECTIVE FUNCTION 


A major task in the development of realistic linear-programming models 
is the gathering of aecurate and reliable numerical values for the coef- 
fidents. Hence it is important to study the behavior of solutions to à 
linear-programming problem when the coefficients of that problem are 
allowed to vary. This type of investigation is the function of parametric 
linear programming. We can vary the coefficients of the À matrix, the 
objective-funetion cost coefficients, or the constants of the right-hand 
side of the equations. Not much work has been done on the first of these 
alternatives, and only a special case of the last two types of variation has 
been studied in some detail.! "There are still many problems to be solved 
in this area. In what follows, we shall be concerned only with the 
material contained in the references cited in the footnote below. 

'The investigation of parametric programming as applied to the varia- 
tion of the cost coefficients originated in the study of the production- 
scheduling problem described in Sec. 1 of Chap. 11.? There weintroduced 
the parameter ^, which measures the cost of a unit increase in output 
relative to that of storing a unit for 1 month. Before selecting a particu- 
lar production schedule that 1s optimal for some specified value of ^, the 
manufacturer might find it advantageous to investigate a set of optimal 
solutions corresponding to ranges of values of ^. In this manner he can 
more accurately select the solution that not only best fits his production 
and storage capabilities, but also suits those factors that have not been 
incorporated into the linear-programming model. "The procedure that 
enables one to compute efficiently the associated optimal solutions for 
ranges of À is described below. "This computational scheme is basically 
a variation of the standard simplex method. 


! See Gass and Saaty [48] and Manne [74]. Credit should be given to W. W. 
Jacobs for initiating the early work in parametric programming and establishing the 


underlying concepts. 
? The reader not familiar with this problem should first review the cited section. 


136 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 


Mathematically, the problem can be stated as follows: Let à à X 9, 
where à may be an arbitrary, algebraically small, but finite, number and 
o may be an arbitrary, algebraically large, but finite number. For each 


À in this interval, find a vector X — (xizs», . . . ,z,) which minimizes 
Y, (d; - Mjx;j ü sS 
focii 

subject to 
» Qi; -— bi $-— 1l 205 uc p Wii 
Deed (1.2) 

z; 20 Jie es dd 


where d;, d, a;j, and b; are given constants. 

We assume that our problem 1s nondegenerate and that we have 
available a basic feasible solution of (1.2). Using the simplex method, we 
can solve our problem for À — 6 and obtain either a solution (Case A) or 
the information that the objective function (1.1) with À — 6 has no finite 
minimum on the convex set defined by (1.2) (Case B). 

Case A. Since in our problem the cost coefficients c; — d; 4- Ad;, 
we can for any basis represent the terms 2; — c; as a linear function of 
^. Let us write these linear functions as 2; — cj — oj 4- AB;j. "Then, since 
we are in Case A, 

oj; 4- 98; € 0 ques T. los UM 
which means that the inequalities 

oj 3 A8; & 0 re Phan (1.3) 


are consistent. For all 8; «€ 0, we have 


Ac 
o LS 


and for all 8; » 0, 


aj 

max — — 

X'm s 50 CH 
[j Or 


. [27] 

E Inn 

À E Bj;20 j 
or 


T* if all 8; € 0 
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Hence, by the simplex method, the current solution will yield the mini- 
mum for all 4 saüsfying (1.3), i.e., for all A such that 


ACSONUSON 


If X — -Fo, our problem is over. Assume then that À is finite, 
à — —oay/B, for gy 2» 0. If all the corresponding za X 0, then we know 
from the simplex method and the definition of À that our problem has no 
minimum for » À; thus we are finished.! If at least one zi 7 0, then 
the simplex process introduces the vector P, into the basis and eliminates 


a veetor P; in the usual manner. Note that 


zn 0 (1.4) 


Theorem 1l. T'he new basis yields a minimum for at least one value of &. 
If M X X X X ds the entire set of values of X for which the new basis 
yields a minimum, then N' — X. 


Proof. "The basis 1s certainly feasible, since we have followed the simplex 
prescription. Further, the inequalities (for the new basis) 


aS SU qd .—v (1.5) 


are consistent. Forif we imagine ^ — A, we have inserted into the 
basis the vector P; whose z; — c, — oy 4- A8; — 0. The new basis 
will still be a minimum for A — 2, since à satisfies (1.5). 

All that remains to be shown is that À « À does not satisfy (1.5). 


We have 
üp m - 
d (1.6) 
Bx 
/^— TM 
[o em m 


In order to satisfy the inequality from the set (1.5) which corresponds 
to the eliminated vector P;, we must have 


o; 4 MEO 
Or, by (1.4) and (1.6), 
—oy — M3. & O 


Since 8, » 0, this last inequality forces the new range of ^ to be 


1 As in the general simplex method, the column veetor X, — (xus . . . ;Zm&), 
where X, — B-!P, for the current basis B. 
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In this manner, we proceed from one range of values of À to the 
next until we include A — $4. To prove that this process is valid, we 
have only to assure ourselves that no basis is repeated. This assur- 
ance is provided by the following remark: If X is replaced by À 4 e, 
where eis any (conceptually small) positive number, the steps followed 
would be precisely the same as we have already described. The 
nondegeneracy assumption guarantees that we will either solve the 
problem for A — À -F eor obtain the information that there is no mini- 
mum for that value of 4. Hence we cannot remain indefinitely at à 
valueAsuchthat^ —A — X. After we leave a basis, we cannot return 
to it or to any basis corresponding to lower values of 4. By our 
theorem, these intervals of À will not overlap. Itisentirely possible 
(indeed, it has frequently occurred) that, corresponding to some 
solutions, A — X. But, as we have seen, this cannot persist indefi- 
nitely through the various changes of basis. The various ^ and A that 
arise are called characteristic values of X, and the minimizing solutions 
corresponding to the various values of ^ are called characteristic 
solutions. 


Case B. Here, as we attempt to find a minimum feasible solution for 


^ — 6, à vector chosen to go into the basis cannot do so because all of its 
elements rj are nonpositive. "There are two possible situations: 


je 


Let P, be the vector that cannot be introduced into the basis. Here 
we are given o, 4- 08, 7 O and all zu X O. If 8, — O0, then the prob- 
lem has no finite minimum solutions for any A. 


. If 8, « 0, the inequality o, 4- A8, » 0 will hold for all 


A «AX - —(a/fy) 


and hence there will be no finite minimum feasible solutions for ^ in the 
region à X A « NM. At this stage, we do not know whether a finite 
minimum solution exists for Mj. If all a; -- A,8, € 0, we have a mini- 


- à ^ / : 
mum feasible solution for X,, and ^; ean be determined by 


^; 7 min (—aj/8;) 
8j;20 
The charaeteristie solution holds for Aj X ^ X Xi, and the pro- 
cedure of Case A ean now be applied. If not all o; 2- M8; € 0, any 
vector having o; «- M8; » 0 can be introduced into the basis. "This 
eriterion is used in the following iterations until all the transformed 
aj -- xfj € Ooruntil a vector with o, 4- X,Bi 7» 0 cannot be introduced 
because all its transformed elements are nonpositive. The former 
condition can be handled by the method of Case A. In the latter 
condition, if 8, 7 0, no finite minimum solutions exist. For pex, 
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we know there are no finite solutions for A « M; — — (o,/8,), where 
A; M. We now attempt to determine whether a finite solution 
exists for A — Xj. Suceessive applieations of the above procedure 
either will lead us to a finite minimum for some ^ (and then Case A can 
be used) or will show that there are no À for which a finite minimum 
exists. 

If we are given a characteristic solution for an arbitrary range 


AO OAUS Nu 


we can proceed to the right of ^;,1 by the procedure of Case A. We 
can proceed to the left of ^; by introducing the vector P, which has 


If all z4, € 0, then there are no finite solutions for À « X; 


Summarizing, we have seen that 


1. By à modification of the general simplex procedure, it is possible to 
investigate systematically and solve the one-parameter objective-func- 
tion problem. 

2. Given any finite minimum solution, we can determine a set of char- 
acteristic solutions and the associated characteristic values for all 
possible values of the parameter. 

3. A solution is minimum over a closed interval of ^. 

4. The set of ^ for which minimum solutions exist is closed and connected. 


Many large-scale linear-programming problems with a parametric 
objective funetion have been solved using an electronic computer. A 
typical problem consisting of 33 equations and 65 variables was solved 
for all positive values of the parameter in 53 iterations. The complete 
solution included 23 characteristic solutions and corresponding charac- 
teristic values. Multiple solutions were generated but not tabulated. 
All problems considered had a finite minimum for ^ — 0, and the range 
of interest was 0 X ^ «€ c. Asacomputational aid, it wasfound advisa- 
ble to solve for the à. — 0 solution using the regular simplex procedure and 
to use this solution to initiate the parametric process. 


Example. Minimize 


ATEM 
subject to 
35—y 25 (1.7) 


EDUC pete 


140 


METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 


where 
c——. xm S NE doe oq 


We transform system (1.7) to a set of equalities with nonnegative 
variables, with one artificial variable »v with associated large positive 
cost coefficient w, to obtain the system (1.8): 


| 
e 


Sxy 23) — (Qn — s) — 9 spopodecs 
2(zxi — x2) -- (y1 — y») -cF us -—-3 (1.8) 


with the corresponding objective function of 
A(zxi — x2) — (yi — V») -]- wv 


Step I of Tableau 8.1 is the system (1.8) in the usual simplex 
tableau. Since there is a natural unit vector, P;, in the system, we 
need to employ only one artificial vector, P;, with large positive 
weight, w. The first basisis (P;P;). The 2; — c; elements are writ- 
ten as oj 4- A8; J- wy; and are entered in the (m 4- 1)st, (m -- 2)nd, 
and (m -J- 3)rd rows, respectively. For example, vector P; has 
21 — €; ^ —A -F 3w; hence we enter 0, —1, 3 in the corresponding 
rows under P,. The value of the objective function is 2 — 5w. 
Since P, has the largest positive value in the (m 4- 3)rd row, it is 
introduced into the basis, and P, 1s eliminated (Step II). Similarly 
we eliminate P; by P, to reach Step IIT. Now, since all the elements 
in the (m 4- 3)rd row equal zero, we have our first feasible solution. 
The basis vectors are P; and P, and the solutionis z; — 85, ys — 1$; 
22 — y12 u— u; —0. The value of the objective function is 
pm e 98. 

Since we first wish to determine whether a finite solution for 
à — AXexists, we should next introduce a vector with a negative element 
inrow m c 2. 'Theonly oneis P;. However, all z;5 € 0, and hence 


there are no finite solutions for A « M — —os/Bs — —2. Applying 
the procedure of Case B, we see that all o; 4- 418; X 0, and Step III 
gives à minimum for —2 XA €3 (M - —a;f&). Introducing 


P, — P, into the basis (we should expect a solution for ^ 7 3), we 
eliminate P; and have a new feasible solution with P4, and P, as basis 
vectors. Thesolutionisy; —5,u3 —28,2—5;12; — 23; — y1 — u, — O. 
(Note that here the value of the objective function is independent of 
^.) Applying the algorithm of Case A, we have P, — P, and 
^i — À$ — 9. Ps cannot be introduced, since all zi; X 0, and we 
have no bounded solutions for A » 3. Therefore, the problem has 


only one characteristie solution for —2 « A « 3 and a multiple 
solution for ^ — 3. 


Sec. 2] PARAMETRIC LINEAR PROGRAMMING 141 


Tableau 8.1 


I. Equations (1.8) in Simplex Tableau (with Artificial Vector P;) 


] 
EN 


l 
7 

3 
"i 


$ 
$ 
0 
9$ 
M 


N- 


tON (MN CD [NN 


l 
Ne NS 


] 
NN 


C UN GONX|oN coN 
NM NOS 


EN 


iS 


C oN cix 


] 
Seresps| Ne 


NOO NE Nn 
C ow GV |oN ox 
] 


An application of the above technique to a problem from the 
theory of games is given in Chap. 12. 


2. THE PARAMETRIC DUAL PROBLEM 


Associated with the parametric objective-function problem is the following 
general dual interpretation, i.e., where the parameter is contained in the 
right-hand side of the equations: Let c X 0 X p. For each 60 in this 
interval find a vector 


Xm (rj$5 v2) 


142 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 


which minimizes 


LU 
M C51; 


J71 
subject to 
, agr c b. 6b; $2 1,2,.. 3m 
jz1 
and 
2x0 ] — 1,2, mn" 


Assume that we have found a minimum feasible solution X — (21,25, 
. ,4) for the problem when 0 — e. We can write each z; as a linear 
combination of e and have 


i;— qid cp; 20 
which implies that the inequalities 
dS; 0p 2 0 (2.1) 


are consistent. 

If all p; — 0, then the solution X is a minimum feasible solution for all 
0; if all p; 2» 0, then the solution X is a minimum feasible solution for all 
0 — c; and if all p; X 0, the solution X is a minimum feasible solution for 
allo X ec. In general, however, the p; will be both positive and negative, 
and in order to determine for what range of values of 6 the given solution 
X is à minimum, we must perform an analysis similar to that performed 
for the parametric objective-function problem. 

For p; » 0, we have 


e» 
Pi 
and define 
max — 2 
0 L— pi»0 Di 
E Or 


eos if all p; € 0 


For p; « 0, we have 


(drums 
Di 
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and define 
min — zu 
0 -— pO Di 
or 


nuls if all p; 2 0 
The given solution (2.1) is then à minimum for 
90s 


We shall assume that the upper bound 8 is not 4- ». As 6 is increased, 
the solution remains optimal, i.e., each z; — c, X 0, but it need not stay 
feasible. 

For a sufficiently large increase in 0, one of the values 


d; qi 4 Opi 


will be forced negative. If i, is the first of these variables to go negative, 
then 

ecd 

Di 

where p; « 0. We next wish to determine a new minimum feasible solu- 
tion for a range of 0 2 0. We have to determine a vector to be intro- 
duced into the basis and a vector to be eliminated that will keep the 
right-hand elements of the transformed equations nonnegative and the 
transformed 2; — c; nonpositive. The method of selection 1s summarized 
as follows: 


Theorem 2. If the vector P, corresponding to 8 — —qi/pi ds eliminated 
from the basis and a vector Py with 
4 — €. nin S S 
YXk zijg0 — Vl 
is introduced «nto the basis, the new solution is a minimum for at least 
one value of 0. If 0' «X 0 X 0' is lhe entire set of 0 for which the new 
basis yields a minimum, then 6' — 8. 


Proof. The new solution X' — (2,25, . . . ,2;) is given by 
8| qp — up — (nop) — foris 
[ 
Qm pur un 


X Uk 
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The new solution is certainly feasible for 8 — 8 — —qn/p. If X318 
feasible for any other 6, then it must be for 6 2 60, since zi, « 0, 
p; « 0, and 


qu 0pi 560 
Vu zs 
implies 
g»—3S-g 
pi 


To show that the new basis is optimal, we have 


z 
(pe d cose E — €) 


Sinee all 2; — c; € 0 and zi « 0, all (z; — cj)' with an zi; 2 0 are 
also nonpositive. In order to have 


U Pr 
edm Lr rect D 


for zi; « 0, we must have 
ect ue) 


or 


Hence P, must be the vector not in the basis that corresponds to 

. Rj — Cj 

min — : 
zjc0 WX 


proof is left to the reader (see Sec. 2 of Chap. 9). 


If all zi; 7 0, then there are no feasible solutions for 8 8. "The 


3. SENSITIVITY ANALYSIS 


Except for the discussions in Sees. 1 and 2, we have assumed that all the 
coefficients of a linear-programming problem are given. However, for 
many problems either these constants, i.e., the aj, b;, and c;, are estimates 
or they vary over time. For example, in a diet problem such as the 
chieken-feed problem, the cost of any individual feed will vary from week 
to week, and it is quite Important to know the range of cost for which the 
solution remains optimal. Similar situations may arise for other ele- 
ments of a problem. Investigations that deal with the changes in the 
optimum solution due to chànges in the data are termed sensitivity 
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analyses. In this section, we shall be concerned with the analysis that 
determines the range of a given element for which the solution, as origi- 
nally stated, remains optimal, and we eonclude with a discussion of the 
sensitivity of the objective function with regard to an implicit parameter 
in the neighborhood of the optimal vertex. Additional work in sensi- 
tivity and related areas is given in Shetty [90a], Courtillot [16a], Garvin 
[45a], Madansky [72a], Wagner [99a], Saaty [88a], and Webb [100a]. 

a. Variation of ihe cj. For an optimum solution we have the set of 
inequalities 2; — c; X 0 holding for all j. 

Let Ac; be the amount to be added to the corresponding ej. For those 
variables not in the final basis, we must have z; — (c; 4- ^cj) € 0; hence, 
2; — cj € Acj, with Ac; having no upper bound. An appropriate change 
in c; does not change the value of the objective function since v; — 0. 
For the example of Tableau 4.3, we have for those variables not in the 
final solution that —Ll$ € Ac —41$ € Ac, and —126 X Ac;. 

For a variable in the final basis, a Ac; affects all z;, for all j not in the 
basis, since 

2; — 6; — » zoo 

iin basis 
Let the change Ac; occur for some basis variable z,. "Then 
2 X1C;i -- TD RTANE- EC X 0 

i in basis 
Or 

XyjACc, € —(2, — cj) 


For those z,; 7 0 


ACk - -0 A Cj) 
ES Zu 


and for those z;; « 0 
(s — c) 


Ac, 
Xkj 
Hence 
—(z; — ec; . —(8-—ec; 
max et Ea X Ac, X min esee 
rk;«O0 Tk; zkj20 Xkj 


for all j not in the basis. If no z,; 7» 0, there is no upper bound, and if 
no z,; « 0, there is no lower bound. For the example of Tableau 4.3, 
let Ac, — Acs.. We have 
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For Ac, — Ac», we have 


. (Vi 46 126 
mS 0 S DA S " 


A 
755: 1:075 


EE 


With Ac, restricted by the appropriate bounds, the basis remains optimal 
but the value of the objective function changes by Aciz;. 
b. Variation of the b;. A change in a b; must be of the magnitude 
that preserves the feasibility of the basis. For the optimum solution, 
? — B-!b 7 O, and for a change Ab; in bj, we must have, letting b be the 
new right-hand side, 


Xo- B-!b I (zi -L- bab) 20 (8.1) 


for all 4 in the basis and where b;; is the element in the ?th row and ith 
column of B-!. 
For bà 7 0, we have 


— 4 
bic 
$ i ys bi 


and for b; « 0 


qus 
Ab, X : 
[nm 
Hence, 
—*i mast 
max SS AbDy mum 
bu»0 ba buco il 


For the example of Tableau 4.3, B — (P,P;PB4), X? — (4,5,11), 
b — (7,12,10), and 


^j Me 0 


Z5 
BO-( o 0 
jen 


Letting Ab, — Ab; yields 


T ese. en) ; —11 
max (c. r3 X Ab. X Y 


7:010 500. 


e NUT 


The new solution is given by (3.1), and the change in the value of the 
objective function is given by » bi Abic;. 


i in basis 


c. Variation in aj. Let aj, the element to be varied, be the element 
of the ith row of column P,. Assume that P, is a vector of the optimum 
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basis and that the optimum solution is X? 2 B-!b. The new matrix is 
given by B — B 4- Aaj Oy, where Oris. à. null matrix except for element 
(Lk), whieh equals unity. "To preserve feasibility (assuming that B is a 
nonsingular matrix), we must have 


B-!b - (B 4- Aay O5)? b 2» O (3.2) 
and to preserve optimality 
(Cc E c,B-1P; L6 eu (3:19) 


for all 7, where co is the set of cost coefficients for the basis variables. 
We have 


B - B(I 4- B-! Aa; Oi) 
and, by Sec. 1 of Chap. 2, 
B-! — (I -- B-! Aa, Oj)-!B-7! 
Equation (3.2) can then be rewritten as 
B-!b — (I 4- B-! Aa, Oi)-!B-!b 
— (I -- B^! Aay Oi) ?X^ (3.4) 


We now must determine for what conditions on Aa; the inverse 
(I 4- B-! Aa; Oi)-! exists and restricts (3.4) to be nonnegative. 
'The term B-! Aa;; Oy, can be written as 


B^! Aay, Oi, — 
ON EO cni DeeeT oo Nu sauce I) 
ue HM UU NI mU Me ME 
NORD au 
kth column kth column 


where the bj; are the corresponding elements of the ith column of B^. 
Hence, 
(I 4- B7! Aa, Ou) — 
1 ( m n bu AQ rete 0 


0 vc SEDSLAmR 9 0 pe-hih row- (3.5) 


0 Bom bs Aa ECeDCS 1 
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The inverse of (3.5) is given by 


1 |cbuAau 0 

l 4- bu Aa 
1 
d Aero eus DECEPTUS 3.6) 
(I -- B^! Aay Ox) 0 i35 Ad ( 

— b, Aau 

MM ERRARE 1l 
i 1l -F 54 Aa 


Substituting (3.6) in (3.4), we obtain 


B-!b —^ (I 4- B^ Aay Oy) !X? 
Em bj; Aau " 
à l 4- b, Aa : 
B-b- E (3.7) 
1l 4 54 Aau 
b, Aa 
Xm 


1-4 bà Ann A 


In order for (3.7) to exist and to be nonnegative, we must have the 
following: 


1-4 b Aay 0 (3.8) 
bi Aan 
Xi — ICD ADU zs 0 (3.9) 


Solving for Aau, (3.9) yields (assuming that 1 4- b; Aa; 7 0) the con- 
ditions for all 4 z& k: 


HS 
Aun eec POSEE UY 
Qik bazi -€ Dub. for b Ik buzi Ex 0 
Xa 


bury, — buxi Que for bac. bx «0 


The upper and lower bounds do not exist if corresponding denominators 
donotexist. If the change Aa; is for a vector in the basis, the feasibility 
conditions impose the restriction (3.8) and 
Xi - Xi 
max X Aay X min - 
sp (Dux. — bam) « 0 — — ^ 1 (ug. —- Bas c0 


(3.10) 
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For those vectors P; not in the basis, the optimality conditions (3.3) 
must be simultaneously satisfied. Since B-! — (I 4- B-! Aaj Oi) B-!, 
the above analysis yields for (3.3) 
cXI-FB- Aon On -B-P —6us0 (a 
or 
co(I - B-! AQ On)-!X; eccL (en X 0 (9.12) 


where X; — B^ P; From (3.6), inequalities (3.12) become 


"ee by Aa zo 
: Dou AU; 
1 
um MIN E e 
z l 4 ba Aa x: aped 
P bs Aa " 
iig lus A. 
"This yields 
bu Aau Jn 1 xd uaa 
xs c — 1-4 ba Aan a) * us l 4- 54 Aau p 


boi A 


c cs (e« eu EET 2 don cuc 


where the term c,zj; — ciz,; has been added to the left-hand side. 
Collecting terms, we have 


53i Aauk 1 
* ^e A e — ——— m x * — . . * E C t . 
torem iue DNE T bua Aai qd 2i lona 
^ 1n b&SIS 
boi AQ 
je cerae icm ;,—0€0 (3.13 
mem mr MES NEU 


By multiplying czj; by 1 4- bu Aai/(1 4 b au), substituting for zj, 
and collecting terms in (3.13), we have 


bi; Aa. 
ees ecd LE e pp ce (0) 
(e; — e) l 4 bi Aaix pres 


1 in basis 


150 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 


and from (3.8), 
(a; — c) (1 - ba Aau) — Aay Y, bacis; € O0 


(2; -— Cj) 4 (2j -— cj)bii AG -— AQ B baci e 0 
(e E Cj) X Aau P» DuC (2; — cj)ba ] 


If the term in brackets 1s positive, 


PERPE, X Aa 
bi buciby — (ee cj)bu 
and if it is negative, 


2j c6; 


E 
by bacixk; d (2j ar cj)b 


AQ 


or, for those j not in the basis, 


£3 — 6j 
max SE Aa 
j [ 25; D bac; (en cj)bu] 2 


: 2;—c 
X min : - 


.14 
9j | zs; 2 buc; — (25 — cj)bu ] «e G | 


The Aay is unbounded if corresponding denominators do not exist. We 
note that the summation term is the ith element of the row vector c;B-!. 
In order to apply a change Aay to the ith component of à vector P, in 
the basis, Aa, must satisfy (3.8), (3.10), and (3.14). "The corresponding 
changes in the value of the variables are given by (3.7). "The total change 
in the value of the objective function is determined by subtracting the 
value of the old solution X? from the value of the solution (3.7), i.e., 


, bu Aag em Cie 
(o Y Fg An a) quy i l 4 bu ^au 


220 byi Aau. zx ]c eux Cic Cs z 
ME Rut dui EE Lx edu M m ac ue E 
"o ]l-4baAaux i : 2s oli 


Adding the term 
bj Aa Mp bj Aag. B 
l-cbgAe ^ — lc wAARk 7 
and collecting terms, we have 


: "7 1 4 5a Aag 
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For a change to an element a; of a vector P; not in the basis, aj; must 
be such that 


c;B^?P,—c x0 
For a given Aajj, we have 

co(X; T Aa; B) —c6; € 0 (9.15) 
where B! is the /th column of B-!*. "Then (3.15) is 

(e; — 65) -- Aag ? bac; & 0 


Hence, for the vector P; not in the basis, Aa;; must lie between the limits 
0) x20) 
——À— — X Aa S uu——————— 
(2 baci) d 0 i 26s bac;) p 0 
"That is; 


— 9 — d ybw 0 
uci i 
Aa € 2 


T if » brc s D 
uM ELE 
2 baci i e 


EO il o5 zm 0 


Aai j 


IV 


Another approach to investigating the sensitivity of an optimum 
solution to changes in the constant terms is given in Saaty [88a] and Webb 
[100a]. From the statement of the symmetric primal and dual linear- 
programming problems (Chap. 5, Sec. 2), we have 


Y ais; 2 bi DE Nie E 

j 
dz ES. uice 2 c NDS 

f Ycgz;- min 
7 
and 

Y agwi € 6j 13712, . ,T 

w;20 1-1, 2; , Th 


g^ Y b;w; — max 


152 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 
We note that f — c)B-!b, where B is the optimal basis and c, is the set 
of eost coefficients for the optimal-basis vectors. 

When f is considered as a function of a;;, b;, and c;, and these elements 
are considered as functions of an implicit parameter !, the total deriva- 
tive of f with respect to (1s given by 


df | N f day | V 9f de; 9f db; 
di — £ dag dt ' Z4 on di — £L b. di 


jl j71 d— 


From [88a] the total derivative can then be written in a computable form 


as 
, »: 

Letting X^ be the optimal vector for the primal and W? be the optimal 

veetor of the dual, we have, based on the duality theorem, 


Of | xo Dd 
IMPOR 


LUN 


df — v Of 9f da; 
dt Te 0b; 96; dt 
12,9 


9f LINT 9f dbi 
c; qu z »-. dt 


(3.16) 


—Wo 


In (3.16) all the partials and derivatives can besolved explicitly. "The 
formula provides us with the sensitivity of the objective function with 
regard to an implicit parameter in the neighborhood of the optimal vertex. 

As an example, consider the linear-programming problem of Tableau 
4.8. We shall assume that the coefficients are multiplied by powers of t 
as follows: a;j?, bit, cj. We are interested in the rate of change of the 
objective function with respect to t in the neighborhood of the optimal 


point. 


Computing all the partial derivatives and derivatives, we have 


JT--4— X--4 

3 a $3 -s5 J-u 

de gn d e 9n d c ep 

SET leds e^ o 10 

E E E eb E - —2t E 4t 
m - —8i zo - 6 I: - 16i -: - 2t 


Sec. 3] PARAMETRIC LINEAR PROGRAMMING 153 


and the total derivative 
d - -(-M65 — (-490)(-20 
-C-39)(-40 — (50 0)(80 
4) G0) -- (5)(—90) 
Cree E462) 
z-ddfoE cq 


The last expression indicates the rate of change of the objective function 
with respect to a change in (. We note that for a small inerement of 
we can estimate the effect on the objective function, i.e., 


Af — —33P At -- 221 At — T11AL 


For instance, if we evaluate the effect of At in this example (where ( — 1), 
we have 


Af —- —22At 


One must be sure that At 1s chosen small enough so that the old optimal 
basis 1s still the solution. 

In Webb [100a] an extension based on the derivation of df/dt 1s given. 
It is shown that 

Ru 


055.0 
E cce be 
0G; : 


In addition, by letting & — c; in the derivative df/dt, we have 


eh 
DE 
and by allowing £ — b; 
CL NN: 
db; 


Two additional important relationships can also be developed. The 
variation of the primal solution with respect to changes in a;; when 
evaluated for a particular zj?, 1s given by 


02 FE VEDETIE 
0aij ro Ci 


and the variation of the dual solution with respect to changes in aij 
when evaluated for a particular w;?, is given by 


Qui EM mop 
0j |u;^ bi 


154 METHODS: THEORETICAL AND COMPUTATIONAL [Chap. 8 


Extensions of this technique into statistical distributions over a multi- 
plicity of optimal solutions are given in Webb [100a]. 


REMARKS 


For additional reading the reader is referred to Saaty and Gass [89] and Barnett 
[3e]. 


EXERCISES 


1. Solve the following parametric objective-function problem for all values of ^: 
Minimize 


2Xz; J- (1 — X)zs — 8z3 -J- Ax, 4- 2x54 — 3Aze 


subject to 


zi Oxs — X3 -F 2zs ze 
—2z» -- Axa 4- x4 912 
—4r5 4- 325 -- 8z; 4- Tg zo) 
and 
Tj z 0 


2. Write the dual to the problem in Exercise 1 and solve for all values of the 
parameter. 
3. Solve the production-scheduling problem (Exercise 1, Chap. 11) with the 
4 5 


objective function 16s; 4- p S; »y y: to be minimum for all values of ^ — 0. 
PE :cm 
4. Modify the computational procedure of the revised simplex method to handle a 
parametric objective funcion. 
b. By means of thesimplex method and a two-dimensional graph of the parameter 
space, solve the following two-parameter problem: 
Minimize 


X; -L Nez Fo uXsc-b o Xa-b S&-b Xe XT A€a -- ucs 


subject to 
Xi — d — 2rs Jd 4x; J4-2xs d 794 —4 
p» 2x, — 2xs& JF rs 4- 2x4 -L 4Axg — 
X3 spent oy ere dp T2, — l 


For this problem a feasible solution exists for all values and combinations of ^ 
and u. Construct an example that does not have this property. What is the dual 
analogue for the two-parameter problem? — Can an efficient computational scheme 
be developed for the n-parameter (n 2 2) problem? 

6. Describe procedures that will allow for the addition or deletion of a constraint. 

T. Generalize the formulas for varying a single b; to include variations in all b;. 


8. Generalize the formulas for varying a single c; to include variations in all c; for 
the set of basis variables. 
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9. Apply formula (3.16) to Exercise 1 when À — t?, each a;; is multiplied by t — 1, 
and each b; is multiplied by t? — t 4 1. 

10. In the corresponding optimal solutions, determine the range of the variation 
for the elements in the following problems: 

a. In Exercise 1a of Chap. 4: 

l. The cost coefficient c;. 
2. 'The cost coefficient c;. 
3. The right-hand-side coefficient b;. 

b. In Exercise 1c of Chap. 4: 

. The cost coefficient ci. 

. The coefficient a». 

. The coefficient a5. 

. The right-hand-side coefficient 5;. 
5. The cost coefficient c,. 

11. Let the optimum basis to a linear-programming problem be denoted by 
B-(PBi:--P,--- P,). Let P, be changed (perturbed) to P.. Determine con- 
ditions on P, such that the B' 2 (P, - - - P, - - - P,) istheoptimal basis (Barnett 
[3e]). Similarly, for a vector P; not in the optimum basis, let it be perturbed to P;. 
What are the conditions on P, such that B remains the optimal basis? 

12. For the linear-programming problem minimize cX subject to AX — b, X ^ O, 
we have determined the optimal solution with corresponding basis B. Consider the 
new problem AX — b with b; — b; -- 6foralliandallvaluesof 6. Develop a formula 
which tells for what range of € the basis B will be an optimum basis for the new problem. 
Note that the basis B is optimum for 0 — 0 and b — b -- 0 where 0 is a column vector 
with elements all equal to 6. 
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n 

13. For the parametric dual problem of maximizing » cjr; subject to 
j71 
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2072280) all j 
prove that the objective function is à concave function of 6.1 Also, show for the 


n 
parametric primal problem with the objective of minimizing » (dj 4- Ad)r;, that the 


s 
n 


objective function is a convex function of X. Hint for dual: Let z;(0) — max D Cj2j 

j-1 
and show that aze(01) 4- (1 — a)zo(02) € 2o(0;.), where O0 X a € 1, 61 «€ 6», and 
05 -— a0; -F (Gl ——- o) 05. 


1 See Chap. 13 for definitions of concave and convex functions. 


CHAPTER 9 / ADDITIONAL 
COMPUTATIONAL TECHNIOUES 


In this chapter we shall consider the question faced by the investigator 
in the field who has formulated a linear-programming problem and deter- 
mined the necessary coefficients and is ready to solve the associated 
numerical example. JWAat preliminary analysis and computational devices 
should he use im order to solve ihe problem with the minimum amount of 
computing? 

One reason that the above query is an open question is that the 
mathematical model of a specific application may lead to a simplified com- 
putational scheme. — This is the case, for example, for the transportation 
and the production-scheduling problems discussed in Chaps. 10 and 11, 
respectively. In the former, a computationally simple variation of the 
simplex method is devised, while in the latter we could either reduce the 
number of equations in the system and solve it by the general simplex 
procedure or solve it by even simpler techniques. However, in the final 
analysis, it may prove to be more efficient not to worry about such 
schemes, since investigations of such possible variations, although inter- 
esting, are in themselves time-consuming and may be unfruitful. Hence 
we shall preface our discussion by emphasizing that any linear-program- 
ming problem ean be solved by the tried and proved techniques of the 
standard original and revised simplex procedures described in Chaps. 4 
and 6, respectively. Since the development of reduction and new com- 
putational methods is dependent upon the specifie mathematical equa- 
tions under investigation, we are here necessarily limited to discussing 
techniques for reducing the amount of computation that are applicable to 
most linear-programming problems. "These schemes are all based on 
overcoming certain peculiarities of the simplex procedure and are designed 
to reduce the total number of iterations required to reach the minimum 
solution. Some of these are described in Sec. 1 below, but let us first 
review a few pertinent points of the simplex method. 
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The general simplex method is divided into two distinct computa- 
tional phases: Phase I 1s concerned with determining a first basic feasible 
solution, while Phase II, which starts with this first solution, is concerned 
with obtaining a minimum feasible solution. Computational experience 
has shown that it takes approximately m iterations (where m is the number 
of equations in the final model) to solve the problem if the computation 
starts with an explicit basis of m unit vectors; approximately 2m iterations 
are required if a full artificial basis is used. Hence it appears to be 
advantageous to eliminate Phase I altogether, or at least to start with as 
few artificial vectors as possible. Further, if we are in Phase I, the selec- 
tion of à new vector to be introduced into the basis depends only on a 
criterion that consistently reduces the contribution of the artificial varia- 
bles to the value of the objective funetion. This criterion is used until the 
"artificial" part of the objective funetion is made zero. "The criterion 
has no control over that part of the objective function that must eventu- 
ally be reduced to à minimum by Phase II. "This "real" part is allowed 
to fluctuate, and usually the value of the objective function for the first 
feasible solution is far from the minimum value. It seems reasonable to 
expect that the number of iterations required in Phase II depends on how 
close the first feasible solution is to the minimum. That is, the closer the 
first solution, the fewer the iterations required. Whats more important 
is that the artifieial-basis technique does not allow us to use certain 
information inherent in many programming problems. For example, 
from our knowledge of a particular problem, we may expect a specific set 
of m vectors to yield a solution that 1s close to the minimum. . However, 
since there 1s no assurance that this set of vectors forms a feasible basis, 
it cannot, in general, be efficiently employed in the standard computa- 
tional procedures. Again, it 1s felt that the total number of iterations 
will be reduced if we utilize some of our knowledge about the ''expected 
solution."! "The majority of the schemes developed for speeding the 
computation are designed to offset the above points and to vary the 
initial simplex computational scheme in order to obtain a ''good" first 
feasible solution (Gass [46], Orchard-Hays [82], and Vajda [97]). 

Experimental work has been conducted comparing the efficacy of 
proposed Phase I methods, alternative rules for selecting the vector to 
enter the basis, and the alternative forms for carrying out the simplex 
algorithm—standard, revised, and revised-produet form of the inverse 
(Wolfe and Cutler [105g] and Quandt and Kuhn (85c]). It appears as if 


! The cautious wording of the above sentences is due to the existence of the ever- 
present "*counter-examples." Problems have been constructed that start with a full 
artificial basis and reach the minimum in exactly m iterations; problems also exist 
that start with a high value of the objective function and rapidly reach the minimum 
or, conversely, start close to the minimum and "'ereep" to the optimum. 
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the product form using the pivot-selection rule of Dickson and Frederick 
[34a]! offers the best combination for achieving computational efficiency. 
Here, efficiency is measured in terms of total number of iterations, num- 
ber of computations, and computational time. 


1. DETERMINING A FIRST FEASIBLE SOLUTION 


A set of m X n constraints to a linear-programming problem can take 
on three basic configurations. Let us first assume that the final set of 
constraints of the model is given by 


AX — b with b 2 O m «n (b 


We assume at least one b; » 0. 
For the set (1.1) we have three situations: 


1. The set contains no unit vectors, and the computation starts with a 
set of m artificial vectors. 

2. The set contains k distinct unit vectors, and the Phase I procedure 
starts with m — k artificial vectors. 

3. The set contains a basis of m unit vectors, and the computation begins 
with Phase II. 


Next, if the model is of the form 
AX X b with b 2 O (1.2) 


we rewrite (1.2) as equalities in terms of nonnegative variables (as 
described in Sec. 4 of Chap. 2 and Sec. 4 of Chap. 4). The resulting set 
of equations contains a basis of m unit vectors, and the computation 
begins with Phase II. 

A third possibility 1s that the constraints are of the form 


AX b with b » O (1.3) 


! 'This rule is as follows: Define d; — (z; — cj)?/[(e; — cj)? 4- »s z;j*(4-)], where 


the z;;(--) are the positive elements in jth column of thesimplex tableau. "The vector 
P, to be introduced into the basis corresponds to d, — max d; for which (z; — cj) 0. 


j 

Modifications of this rule which are computationally simpler have also proved effective. 
This rule is based on the angle, and thus the projections, the vectors P; make with 
cost axis (see Figs. 4.1 and 4.2 of Chap. 4). Dickson and Frederick noted that, in 
general, the rule required a 10 per cent increase in computation time per iteration, 
but solved their test problems in 30 to 70 per cent less iterations when compared with 
the standard selection rule. 
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Here we also rewrite (1.3) as equations in nonnegative variables, but the 
resulting system contains a set of m negative unit vectors. Let this 
system be given by 


AX —X-—b (1.4) 


where X — (x,,53549, . . . £445) is à nonnegative column vector. By 
applying a simple transformation to the coefficients of X, X, and the 
column vector b, we can start the computation with only one artificial 
vector. The scheme calls for determining max b; — b, and adding the 


sth row to the negative of every other row of (1.4). "The resulting set of 
m equations will contain zm — 1 distinct positive unit vectors and will 
require one artificial vector, which corresponds to the sth variable. The 
transformation that 1s applied to (1.4) is given by 


0, —G;--a;  forizss j-21,2,...,n-4-m 

0, — 0j j51,9,5.. v nzEm 

b. - —b,-- b, fori zs (1.5) 
b.e b, 


To illustrate this scheme, let us consider the inequalities 


Zj;— 25 4-325 — 1 
2x1 — zx» 4 Xs zz) (1.6) 
21-1292 — 2522 


We rewrite (1.6) as equalities to obtain! 


Zi — zx d- 3233 — x4 —3l 
221 zo -- X3 UD —0 (7) 
Ty. — 3a gue 


Here max b; — b; ^ 2, and applying formulas (1.5) to (1.7) gives us 


u 


2*5 — 4x. -- m4 — fg — 1 
—ag; 4 2r5 — 2x3 Tzg—2&422 
Tuc Ese da — $&—2 


1 'The transformation does not have to be applied to those rows which have their 
corresponding b; — 0, as in the second equation, but here we illustrate the general 
application of the transformation. 
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We can, of course, formulate a model that contains a mixture of 
inequalities and equalities, that is, AX Zb, where b is unrestricted. 
Here, by suitable manipulation of the appropriate constraints, we can, as 
above, introduce as many unit vectors as possible. 

Up to now we have been assuming that the computation must begin 
with an explicit unit matrix for the first feasible basis. In Sec. 2 of Chap. 
4 we briefly mentioned the possibility of selecting an arbitrary set of m 
vectors for the first admissible basis. Here the plan calls for the selection 
of m vectors to form an initial "expected solution." This selection 
should be based on a careful, but not time-consuming, analysis of the 
program objectives. We next attempt to compute the inverse of the 
corresponding m X m matrix. 'The following possibilities may occur: 


1. The set of m vectors is linearly independent (the inverse exists). 
a. The corresponding solution 1s feasible. 
For this case the simplex procedure starts Phase II with a preferred 
solution. 
b. The corresponding solution 1s not feasible. 
Here, as will be described below, we can start Phase I with only one 
artificial vector. 

2. The set of m vectors 1s linearly dependent and is of rank k «€ m. For 
this situation we can apply a transformation to the tableau to initiate 
Phase I with either m — k or m — k 4- 1 artificial vectors (see Gass 
(46] for further discussion). We could also continue to choose other 
vectors for our initial basis from a secondary set of preferred vectors 
until we have chosen a linearly independent set of m vectors (see 
Orchard-Hays [82] for additional comments along these lines). 


For Case 1b above, the following simple transformation, which is similar 
to the one applied to the set of inequalities AX 7 b, will enable us to 
begin Phase I with one artificial vector. For ease of discussion we shall 
describe the transformation in terms of the original simplex tableau of 
Chap. 4 (see Tableau 4.1). Let us assume that the first m vectors, 
P, P5 ..., P, have been selected as the preferred set and are 
linearly independent but do not form an admissible basis. Let z;; for 
;—1,2,...,m--1andj 20,1, .. . , n bethe transformed element 
in the ?th row and jth column after the matrix (P;P; : - - P,) has been 
inverted, and let z; — 1 for — 1,2, . . . , m. Here we assume some 


) 


fua 90; deb - min r;. Next, transform all the elements of the 


1 


tableau by the formulas 


Hu 
Il 


TIL M CINE for? zé.s,m--1 


gem (1.8) 
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Formulas (1.8) hold for j — 0, 1,.. . . , n and represent the subtraction 
of the sth row from all the other rows and the multiplication of the sth 
row by (—1). "This transformation yields a basis consisting of m — 1 
vectors of the selected m and one artificial vector whose corresponding 
variable z;, » 0. "The new elements of the (m 4- 1)st row are 


Lu 
"d n * 
Did bns, gi s zs 2, Ci TOT SS 
1 
/ 
m (1.8) 


/ 
VTm-4-l, 8 c rag » Ci 
iM 


The elements in row s of the tableau guide the sueceeding iterations until 
the artificial variable is reduced to zero or until it is determined that 
the problem is not feasible. Row s corresponds to the (m -- 2)nd row of 
the artificial-basis tableau with one artificial veetor. Since we wish to 
control the 'real" part of the value of the objective function, we can 
modify the criterion that selects the new vector for the basis as follows: 
For the tableau given by (1.8) and (1.8/), let vector P, correspond to 
"nix mag 1. (1.9) 
z, 0 

Formula (1.9) states that the vector selected has the ''artificial" part of 
its z; — c; equal to z;; 7 0 and the "real" part equal to 1,1; 8S alge- 
braieally large as possible. "This enteron will consistently reduce the 
artificial part and tend to reduce the real part of the objective function. 
If 


y: / 
Tm dio 
min? e -» forc 0 
Po Yi sk 


then the artificial vector will be eliminated, and Phase II begins. If 
2, is not the pivot element, then the next iteration introduces a vector P, 


with z,,, , - max 1,,,,,; €tc., until the artificial vector is eliminated or 
| rz," 20 


until it is determined that no feasible solutions exist. 

For some linear-programming models the problem might call for the 
minimization of different objectives over the same set of linear constraints. 
This problem of multiple objective functions ean be conveniently handled 
by setting up the basic constraints and solving the problem for the first 
objective function. Once the basic minimum feasible solution for this 
problem has been computed, it is a simple matter to determine whether 
this solution is also a minimum for any of the other objective functions. 
If we were using the original simplex procedure, we would include in the 
tableau an extra row for each objective function which transforms under 
the usual elimination formulas. By looking at the corresponding 2; — cj 
elements, we could determine the additional optimized objective func- 
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tions. If we were using the revised procedure, then the optimality of the 
objective functions could be determined by means of the inverse of the 
current basis and the corresponding new cost coefficients. If the opti- 
mum solution for the first objective function were not the minimum for 
one of the other objective functions, then this first optimum solution 
would be used as the first feasible solution for the new objective function. 

'The dual of this problem, the problem of multiple constant vectors, 
may also arise, e.g., solutions to economic models for a number of different 
bills of goods might be desired (see Chap. 11, Sec. 2). This problem can 
be resolved by the following techniques: Let the original problem be that 
of minimizing 


cX (1.10) 
subject to 

AX - P, (1.11) 

x x GL:12) 


In addition to (1.10) to (1.12), we also wish to solve the following set of 
problems: To minimize 


cX 

subject to 
AX — P,, Died 2 E 
X20 


The problem is first solved for (1.10) to (1.12). In the original simplex 
method the additional P,; are attached to the tableau and transformed in 
the usual manner. If, after the minimum feasible solution for P, is 
obtained, all components of all the Po, are nonnegative, then this solution 
is à minimum for all the Po&,. Suppose that, for some q — r, not all the 
elements of the transformed vector Pe, are greater than or equal to zero. 
To develop a starting solution for this problem, we find the minimum 
element of the transformed vector Po, and apply (1.8) and (1.8). The 
resulting solution will consist of one artificial vector and m — 1 vectors of 
the previous minimum feasible solution. "The problem is then solved in 
the usual manner for Po, and similarly for any of the remaining P,,. 
However, instead of applying the procedure associated with Eqs. (1.8), 
we might look at the situation in the light of its dual interpretation. 

Here we have selected a basis—let ussay B — (PjPs - - - P,,)—com- 
puted its inverse, and determined that B-!P, 7» O and that 


cCBOP;— 06-23-60 (1.13) 
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for all j, where c? — (ccs, . . . ,c,) is à row vector. Hence B is an 
admissible basis for (1.10) to (1.12). We have also determined that B is 
not an admissible basis for the constraints 


AX «P, (1.14) 
in that 
B-!P,, (1.15) 


is not a nonnegative vector. We say, for the problem determined 
by (1.14), that the solution (1.15) is optimal but not feasible; i.e., all 
2; — c; € 0, but not all elements of B-!P,, are nonnegative. The dual 
problem to (1.10), (1.14), and (1.12) is to maximize 


WP, (1.16) 
subject to 
WA € cf (1.17) 


For the dual problem we see from (1.13) that the basis B yields a solution 
Sox Ic 17) 35 that 


W:P;j—ccz0 


for W? — c?^B-!. We also have that, for the dual, this solution is feasible 
but not optimal. A computational technique which enables us to employ 
efficiently the information concerning feasibility and optimality of the 
primal and dual contained in the above problem was first proposed by 
Lemke [70] and is called the dual simplex method. 


2. THE DUAL SIMPLEX METHOD 
To consider the general procedure, let us write our problem as follows: 
To minimize 

cX 
subject to 

AX — b 

X20 

with its dual problem: To maximize 


Wb 


1 The dual problem is described in Chap. 5. 
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subject to 
WA €c 


As we did in the case of multiple-constant vectors, let us assume we have 
selected a basis B — (P,P; : : : P,) such that at least one element of 
B-!b is negative and c?^B-P; € cj for all j. A solution to the dual con- 
straints is given by W? — c?B-! with its corresponding value of the objec- 
tive function being 


aft (2.1) 


We wish to develop à computational procedure for the dual which will 
yield a maximizing solution, and hence, by the duality theorems of Chap. 
5, a minimizing solution to the primal. This procedure must then deter- 
mine a new basis for which 


1. The dual inequalities will still be satisfied. 
2. 'The value of the dual objective function will increase (or remain the 
same)! until the maximum or unbounded solution is reached. 


In this fashion we will always preserve optimality of the primal and, in à 
finite number of steps, determine a feasible and optimum solution to the 
primal. 
Let us denote the rowsof B! by Bi. Hence the m components of the 
nonfeasible solution of the primal are given by zi; — Bj;b for i — 1, 2, 
. ,m. Letzj;, — Bib — min Bib « 0. The vector P;, as we shall see, 


will be the one eliminated from the basis. For those vectors not in the 
basis, Le., for those having W?P; «€ cj, compute z; — BjP;. Let us 
assume that at least one zi; « 0. For the set of z;; « 0 form the ratios 
(z; — cj)/xi. Let 


Ligen ap ee e 
0 — min ^ — au 20 (2:2) 
zijc0 Vg X 
The vector P, is selected to replace P;, and the new basis will yield a solu- 
tion to the dual constraints. "The new basis ls 


B-(Bi--:--: PLajBBPu ccc P,) 


! (The value of the dual objective for the new basis may not increase if the dual solu- 
tion is degenerate. "This is manifested by more than m of the quantities c&B-!P; — cj. 

T This criterion for selecting a vector to be introduced into the basis is equivalent 
to the one used in the parametrie dual problem (Theorem 2 of Sec. 2, Chap. 8). The 
proof of that theorem also applies to this situation. 
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and B-! is obtained by application of the elimination formulas on B-! 
as described in Sec. 1 of Chap. 6. The reader can readily verify that the 
new solution to the dual eonstraints is 


W -— W* — 8Bi (2.3) 
and the corresponding value of the objective function is 
Wb — Wb — 6i; (2.4) 


The new solution to the primal constraints ean be computed by the usual 
elimination formulas or directly from X — B-!b. If all X » 0, then we 
have determined an optimum feasible solution to the primal. If not, 
then we know we have made at least zi; — Bib » 0. For this situation, 
we repeat the above dual simplex process of selecting the vector to be 
eliminated and then the vector to be introduced into the basis, until we 
find a basis that solves the dual and is also an admissible basis for the 
primal or until we have determined that the dual has an unbounded solu- 
tion and hence that there are no feasible solutions to the primal. The 
latter case arises when, in computing the z;; — BjP;, all z;; 7 0. If this 
is true, we have from (2.2) and (2.3) that we can construct a solution 
to the dual constraints for any 0 » 0, since 


(W^? S 0B;)P; -— W?P; rc 0B;P; «x W?P, xS e 


From (2.4) the corresponding value of the objective function can be made 
as large as possible, s&nce zx;o « 0. — This situation is revealed in the primal 
tableau, in that we would have all xj; 7 0, zio « 0, with the implication 
that the /th equation, which has been transformed to à nonnegative sum 
of nonnegative variables, is equal to a negative number. The dual sim- 
plex procedure can be employed as a variation of the original simplex 
procedure or of the revised procedure, since it uses exactly the information 
contained in their respective tableaus. 

To handle degeneracy in the dual problem, Lemke [70] sets up the 
perturbed dual 


(aor MEN Ee EE 


and shows that a scheme similar to the degeneracy method for the primal 
may be established. In the dual, we must have a procedure for selecting 
a unique vector P, to be introduced into the basis. "The situation is not 
unique when the value of 0 in Eq. (2.2) assumes the minimum for more 
than one value of 7.1 For these j, we first compute the set of ratios 
zij/xi. lf there is à unique minimum for these ratios, the index 7 of the 
minimum corresponds to the vector to be introduced. 1f not, we next 


1 In practice, if there are ties for the minimuin, then the ratio that corresponds to 
the smallest index is selected for 6. 
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compute for the tied columns the ratios zs;/ri; and repeat the analysis, 
ete. Inthis manner we select a unique P,, and the problem will not cycle. 

In the above presentation of the dual procedure we assumed knowl- 
edge of an optimal but not feasible solution to the primal, i.e., a solution 
to the dual constraints. This is equivalent to starting the simplex proce- 
dure with a known feasible but not optimal solution and hence eliminating 
the artificial Phase I computations. 'The main advantage to using the 
dual simplex method is that the dual statements of many problems have 
explicit solutions. "These problems are characterized by having a set of 
nonnegative cost coefficients, e.g., the diet problem. The dual con- 
straints are immediately solved by the vector W — O. Here we take 
the identity matrix as the corresponding artificial basis. Then, with only 
minor changes, the dual method can be applied to determine the optimum 
solution to the primal (Dantzig [22]). 

For those problems which do not have explicit solutions, 1.e., which do 
not have all c; 2 0, à number of schemes have been proposed (Dantzig 
22] and Vajda [97]). — The first technique sets all negative costs equal to 
zero and starts with the identity matrix as an initial basis. Here the dual 
method, with the appropriate changes, is applied until the optimum is 
reached. "The corresponding basis will be an admissible one for the 
primal, and the usual simplex procedure can proceed to optimize the 
original objective function. The second scheme constructs the dual 
analogue of the artificial-variable technique and solves the associated 
problem in two phases by the dual simplex method. 

Às an example of the dual method, consider the following problem: 
Minimize 


Xs b cbe 
subject to 
$1. — Z3 d 424 — 2$ — —2 
42 — 23— z4-d- x54 — 1 
a 0 


The dual objective is to maximize 
— 240; "I 25 
and the constraints are 
wi se) 
w»«0 
1 
sur 
—1;- Us. «1 


| 
E 
| 
5 
I^ 
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Àn initial basis is given by 


B - (PP) x i " 


Setting up the primal in the usual simplex tableau, we have the following: 


Since all the z; — c; elements are nonpositive, the basis B is a feasible basis 
for the dual, i.e., optimal but not feasible for the primal. The dual 
solution is 


W*- cB- 0 ) k B 50190) 


Applying the dual algorithm, we see that P, is to be eliminated 
(xj x1 —2) 
and vector P; 1s to be introduced into the basis, because 


es 63 mg reU DE 
0 p r-—— E—3 1 
T13 X15 


Here we had a tie for 0 and selected the first vector. "The new tableau is 
as follows: 


Since all z; 2 0 and since z; — c; € 0, we have determined an optimal 
admissible basis for the primal and dual consisting of vectors P» and P;. 
The primal optimum solution is zx — 3 and z; — 2, and that for the dual 
is w, — —1 and w; — 0, with a common optimum value of the objective 
function of 2. "The reader will note that, for the dual constraints that 
correspond to the vectors in the basis (the second and third constraints), 
equality holds, and since we have an alternate optimum solution with 
vector P;, we also have equality in the fifth constraint. 
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Elaborate and efficient procedures have been devised which enable 
one to combine the original simplex algorithm and the dual method to 
solve the general linear-programming problem. "Two of these are the 
composite simplex algorithm (Dantzig [23] and Orchard-Hays [83]) and the 
primal-dual algorithm! (Dantzig, Ford, and Fulkerson (28]). "These tech- 
niques place no restrictions on the signs of the c; and enable one to obtain 
an initial basie optimal or feasible solution. "The reader is also referred 
to another technique for solving the general linear-programming problem 
called the method of leading variables (Beale [5]). "This procedure also 
offers a method for initiating the computation for any problem. 

All the above techniques are intimately related to the standard sim- 
plex method. Only such methods have proved effective for solving the 
general linear-programming problem (Hoffman, Mannos, Sokolowsky, and 
Wiegmann [61a] and Hoffman [59]). ^ Otheriterative and convergent tech- 
niques that solve the problem but are not as efficient as the simplex 
method include the double-desceription method (Raiffa, Thompson, and 
Thrall [86]), numerical methods for zero-sum two-person games by Brown 
[8| and von Neumann [79], relaxation techniques for solving linear 
inequalities (Agmon [1d] and Motzkin and Schoenberg [77]), and a pro- 
jection method for solving linear constraints (Tompkins [94]). 


3. INTEGER LINEAR PROGRAMMING 


Since its introduction as a tool of applied mathematies, the outstanding 
computational problem of linear programming has been that of finding 
the optimum integer solution to a linear program. The need for this 
type of computational procedure 1s emphasized by the great number of 
problems from the realm of combinatorial analysis and the areas of 
scheduling and production that have been formulated as linear-pro- 
gramming problems. 

À search through the general literature on linear programming yields 
a number of references to problems stated in terms of an integer linear 
program. Many of them are given in Dantzig [27, 27a]. 'The list 
includes the fixed-charge problem (Hirsch and Dantzig [56]), the travel- 
ing-salesman problem (Dantzig et al. (285, 28c]) formulated as a multiple- 
trip integer problem by Tucker [96a], and machine scheduling (Wagner 
[99a], Manne [74a]). Other appropriate diseussions are found in Dantzig 
(275, 27c], Gross [52,52a], and Wagner and Whitin [99e]. Recent work of 
Gomory [515, 51e, 51d, 51e], Beale [5a], Benders, Catchpole, and Kuiken 
[6a], and others has opened the way to a practical solution for a number 
of such problems. "We shall describe only the all-integer procedure 


! The reader is referred to Mueller and Cooper [77a] for a discussion comparing 
the computational efficiency of the standard and the primal-dual simplex algorithms. 
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due to Gomory. Surveys describing a wide variety of integer-program- 
ming methods and problems have been written by Balinski [3a] and 
Beale [56]. 

We should note that the successful application of the transportation 
model with integer availabilities and demands is partially due to the fact 
that each basie solution corresponds to an extreme point that has integer 
or zero values for the coordinates. "This is due to the triangular matrix 
of O's and l's associated with the basis. We might ask what similar 
condition 1s sufficient for the general linear-programming model to have 
an integer solution. For this answer we need to refer to the basic trans- 
formation of the simplex procedure: the elimination transformations. 

If zi; represents the element in the ?th row and jth column of the 
simplex tableau and zi 1s the pivot element, the transformed elements 
are given by 


T5 o-— xg— E Cu TO DE 
B Xuk 
(9D) 


qe eu 
3k 


For the purpose of this discussion we shall assume that the starting 
tableau is all-integer. "We see that à very strong condition for the suc- 
cessive solutions to be integer is for each fraction in (3.1) to reduce to 
aninteger. One unusual way for this to happen 1s for each pivot element 
to equal unity. This is, of course, what happens for the transportation 
problem. "This property is equivalent to the selection of a basis 
whose associated matrix has a determinant equal to 1. As we shall see 
below, this is one factor in the computational scheme for solving the 
all-integer programming problem. 

The integer-programming problem is to find à vector X which mini- 
mizes cX subject to AX — b, X — O, with the added (nonlinear) condition 
that the optimal-solution vector have integer coefficients. The geometry 
of the situation ean be pictured by considering the solution to the following 
problem: 

Maximize 


3zi-- X2 
subject to 


qal ds 


D z— Ax» 
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'The optimum noninteger solution is given by z; — ?5$ 
the maximum value equal to 18. 

"This problem is pictured in Fig. 9.1. The convex set of solutions is 
bounded by the heavy lines, and, as indicated with heavy dots, this region 
also includes a number of integer points. "The problem is to determine 
which of these points maximizes the objective function. 


and z; — $$ with 


FIGURE 9.1 


Since the basic computational scheme to be employed is the simplex 
method, we should review the geometry of this technique. "The simplex 
method starts at an extreme point of the convex set of solutions. Each 
iteration, i.e., the selection of a new basis, determines a new extreme 
point which is a neighbor to the old point (has a boundary segment in 
common). This process continues and, after a finite number of steps, 
stops at the global maximum. We see that, in order to have an optimum 
integer solution, we must also have an associated extreme point that is 
integer-valued. For the problem in Fig. 9.1, the simplex process would 
probably start at the extreme point (0,0), move to (4,0), and finally to 
(285,95). It is impossible to indicate what integer point in the solution 
space optimizes the objective function. 

The question that now arises considers the possibility of changing the 
convex set of solutions in such a manner as to make the appropriate 
feasible integer point an extreme point of the new convex region. For 
example, in Fig. 9.2 we have introduced two arbitrary constraints to the 
problem which do the trick. This technique of putting in cutting hyper- 
planes was used in the past for specific formulations. How to introduce 


these euts systematically for any problem is the essence of the procedures 
developed by Gomory. 
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An early approach to the applieation of the cutting-plane method 
required solving the problem by the basie simplex algorithm, ignoring the 
integer constraints. If the optimum solution is not integer, then a cutting 
plane (i.e., a linear inequality) is introduced which preserves the optimality 
state of the primal problem but not the feasibility conditions. A single 
application of the dual algorithm restores feasibility and yields an optimum 
answer to the reduced problem. If this new solution is integer-valued, 
the processstops. lfnot,anew constraint is introduced and the algorithm 
is repeated until an integer solution is found or until an indication is given 
that no integer solution exists for the original problem (Gomory [51c]). 
This method has been superseded by a more efficient procedure, in which 
the problem, assumed to be given in integers, is transformed by a modified 
simplex algorithm which preserves the integer-value characteristic of the 
complete tableau for all iterations (Gomory ([51e]). "This algorithm is 
discussed below. 


FIGURE 9.2 


We assume for starting conditions that the simplex tableau is all- 
integer-valued and that we start with a feasible solution for the dual 
simplex algorithm but not for the primal. Letting Pi, . . . , P, form 
a feasible basis for the dual problem, the initial Tableau 9.1 is given for à 
minimization problem. 

In Tableau 9.1 zoo is the value of the objective function; some z;o « 0 
for 4 2 1 (solution is not primal feasible); zxoj — 2; — c; € 0 for; 2 1 
(solution is dual feasible); and an additional set of n — m constraints, 
zj,—z;—-0forjem-F1,...,n, with z; 2 0 and cj — cj, has been 
added to the tableau. With this set of constraints, we are now forced to 
determine a basic feasible solution that contains n nonnegative variables 
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instead of the usual m; hence we are given the means by which an interior 
point of the original convex set can be expressed. However, we still lack 
the means to force us into the interior and toward the optimum integer- 
valued point. "This is accomplished by a proper selection of cutting 
planes. 

We now wish to introduce new variables into the solution in a manner 
that will enable us to preserve the integer characteristic of the tableau 
and to move toward the optimum integral solution that 1s contained in 
the convex set defined by the equations of Tableau 9.1. "This will be 


Tableau 9.1 


Cm-41 


Pha 


Zmgl* 7^ 


Ximel t oC 


Yu,maliÓ Co^ Tnm 


accomplished by determining cutting hyperplanes defined as inequalities 
in terms of the nonbasie variables and a new slack vector. Introducing 
the new slack vector into the basis with a pivot element! of —1 will 
preserve the integer tableau and move the solution point closer to the 
optimum integer point. A finite number of applications of these new 
constraints leads us to the optimum integer solution. 


! Weselect, — 1 as a pivot element instead of --1as we are using the dual algorithm 
to change solutions. 
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To determine the form of the eutting constraint, let us consider any 
equation from Tableau 9.1 that has a corresponding z;; « 0, for example, 
the ith equation: 


X19 — Ou A TX, m-- l1 m4-1 - TX m-4-24 m-4-2 T de iA - XinZn (8.9) 


where z441, . . . , z, are nonbasic variables. 

We next rewrite each coefficient of (3.3) as a multiple of an integer 
and a remainder, i.e., in the form 5j -F rij, where bi; is an integer, ri; is à 
remainder, and ^ is an unspecified positive number to be determined. 
'The coefficients can then be expressed as 


X 
Lis py (3.4) 
X Ó. 
cm fno (ESO AN 0 «A 


ES bi sis Urs | IE d rij for all j 


1 


where the brackets indicate ''"nteger part of." If zj/A « 0, then 
[r5;/A] — bij; « Osueh that 6A 4- rij — xg. Notethat, forA » 1, we have 
[1/4] 2 0. 

Substituting (3.4) into (3.3) and gathering appropriate terms, we 
have 


EH M S (s] es ') "s «Ee jb 2 ii 


T (| ACT - auper ome (I ius n«) 9 


x Tl om TI,om d 
T0 A ^ Il e| | ! X 2| Juil — | À s| Xm42 — 


- EH de. — H a) — TL omni Üüom4írXma: do coc 
Forums d- rai (3.5) 


We note that any nonnegative integer values of the variables that satisfy 
Eq. (3.3) will also satisfy (2.5). Such a substitution will make the right- 
hand side of (3.5) a nonnegative number. 

We denote the quantity in braces by z,4; and rewrite that expression 
to obtain 


x Uu l 
—-[£]-, [lH] Z 
jz£basis 


For (3.6) we note that z,4, not only must be an integer but also must be 
nonnegative. "The first part is clear since all variables and the bracketed 
quantities are integers. The second part is established by noting that 
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riy & ^ and, if z,,, were a negative integer, the left-hand side of (3.5) 
would be negative, which contradiets the statement that the right-hand 
side is nonnegative. "This nonnegative integer z,,, is introduced as a 
new variable of the problem. 

Restricting A » 1, the expression for z,,, becomes 


z Tom Xl, m YXIn 
YZnps — EH Tzp | n nea s EE Hipcunpx eo fr SES 5 J5 


WXnps 7 bio i bi, m--1Um-4-1 EE bi, Tubos po cc d Dyadin 


or 


Or 
bio RES bi, m-4-1Xm-4-1 m bi, m2 m-4-2 A proe zi brat T U5--s (3,4) 


Forany A » 1, Eq. (3.7)! is a constraint that must be satisfied by any 
integer solution to the original linear-programming problem. After à 
suitable selection of A, Eq. (3.7) can be used as a cutting constraint. 

We note that b;; « 0 since it was assumed that z;; « 0. Also, since 
we are applying the dual simplex method, some zi; for j not in the basis 
must be negative or the problem is not feasible. By selecting ^ large 
enough, all [zi;/A] for z;; « 0 yields a bij; —^ —1; hence, a pivot element 
of —1 is available and the integer tableau is preserved. However, on 
the basis of the dual simplex transformations [Eqs. (2.2) to (2.4)] we note 
that a small À will cause a larger improvement in the objective function 
than a larger ^ since the change is a direct function of 6;g — [zio/^] « 0; 
that is, the new value of the objective function is given by 


z ; 
X00 — i bio — Zoo d- zokbio — xo d (2x EE Ci) EH 


! For A — 1, and by substituting the value of zi; given by (3.3) into (3.6), a cutting 
constraint can be obtained of the form 


Zaq4. — —fo-F » fjr; 


jz*basis 


where the f are the fractional parts of the corresponding z;;. "This constraint is used 
in a similar fashion to (3.7) to solve the general integer-programming problem, i.e., 
when not all the coefficients are integers but all the variables are restricted to integers 
(Gomory [51c]). "This algorithm calls for the solution of the problem using the 
regular simplex algorithm; if the optimal solution is not all-integer, then a row with & 
fractional value of a variable is selected to generate the cutting plane. It can be 
added to the set of original constraints and the new variable r,,, put into the solution 
by an application of the dual simplex method (Sec. 2). However, A must be specified 
such that it yields à pivot element of —1 in Eq. (3.7) and at the same time causes the 
value of the objective function to be reduced as much as possible. 

The reader will note that sinceb;; « 0 and z,,, 7 0, (3.7) cuts" away the current 


basic feasible solution with z,4; — - - - — z, — 0, that is, forces some z; not in the 
current basis into the new basis. 
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where P,, the pivot column, must be the one such that 


4 Zo vh € 6k 
eomm — e 
zi«O0 - zi 


(A zy — cy — O indicates degeneracy in the dual problem.) 
In order to select ^ in a precise fashion, we note the following: Since X 


must be chosen such that bi — [zi/A] — —1, we have from Eqs. (2.2) 
€k — Ck £k — Ck NEMUS eq cC 
E -— f 
[xi./ X] -—L c eX] bij cn 
Or 
Zky — €& 2j — Cj 
x J J 
—1 ae bij 
Let m; be the largest integer for which 
er —— Ck E eO 
—L o ms 
Then 
—bi; — — EH e T for DIE 0 (3.8) 
Hence, for each j, the smallest À that satisfies (3.8) and allows P, to be 
the pivot column, i.e., yields à pivot element b; — —1, is given by 
A; — —(mj/mj;.  Asthe minimum permissible A must be at least as great 
as the largest Aj we have Agi. — max A; [If Aui — A, — —diu — 1l, 
zij«O0 


that 1s, the pivot element happens to be a —1, then we construct a suit- 
able cutting constraint whose variables include only the nonbasic varia- 
bles of the equation, as this implies a selection of À » 1; see Eq. (3.6).] 

^ is not restricted to be an integer, and for P, we have m, — 1, 
Ak TE, ACH zx Àx; hence bu. -— [Zix / Amin] — —]. 


The above procedure can be summarized as follows: 


1. Assume that a dual feasible solution with an all-integer tableau has 
been determined. 

2. From the rows containing a negative constant term, select a row to be 
used to generate the cutting constraint (the /th row).! If no such 
rows exist, an optimum all-integer solution has been determined. 


! The selection of the row that generates the cutting constraint is tied intimately to 
the proof that the all-integer algorithm converges. Gomory [51e] cites the following 
rules which are consistent with the finiteness proof for selection of a row to generate 
the new constraint: 


1. Always select the first row from the top having a negative element. 
2. Select the rows by a cyclic process; i.e., on the first step look at the first row and 
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3. The pivot column corresponds to the P, whose 


—(z, — cj) — min — (2; — cj) for 
zc 


If no z;; « 0 exists, then the problem is not feasible. 
4. For each j having a zi; « 0, determine the largest integer 


E p —( — ej) 
m; 
5. Define M; — — (xij/m;) for xj; « O0. 
6. Determine Aa; - max Ay f max A — l, set an - 1l. 


7. Develop the corresponding cutting constraint (3.7) and add it to 
Tableau 9.1, with the new equation being the (n 4- 1)st and the new 
variable z,,, corresponding to column P,,, (wheres— 1,2, .. ., 
that is, the index of the iteration). 

8. Apply the simplex transformation to the new tableau using bi; — —1 
as the pivot element. This introduces z,,, as a new nonbasie variable, 
and because the pivot element was — 1, the tableau remains in integers. 
The new row is then dropped from further consideration and the 
process repeated. 


We shall illustrate the procedure with an example due to Gomory 
[51e]. 
Minimize 


10z, m 14r» 4j 2975 


subject to 
8z; d llzs 4- 92; — x4 -— 12 
213; -dF 2z2-- 723 — ds zm 
9zsi-F. Orxz- Oz; — rg — 10 


qu oor a 


if it does not have a negative constant, look at its successors; on the second step look 
at the second row and then its successors, etc. 


3. If the rows are chosen at random, a finite process will result with probability 1. 


The usual simplex rule of selecting the one with the largest negative term is not 
covered by the finiteness rule. In developing suitable computer codes for the all- 
integer algorithm, there has been much experimentation in the quest for a rule that 
is efficient for large classes of problems. Combinations of the simplex rule and other 
rules have worked rather well. The reader interested in developing a code would 
be well advised to consult the latest literature in the field. 
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. / / "T 
The set of equations z, — zi r, — 25, 2, — r5 and the conditions 65850) 
have been added in Tableau 9.2, and row 7 has been added for the cutting 
constraint and column P; for the new variable. 


Tableau 9.2 


[7 


-1|O Ov 
ECCO 


'The second equation of the system is used to generate the cutting con- 
straint. We have 
eb — Ch gj; — 6j zc MI 


—3 080001 — 
-1 ^ne -1 -1 


Hence column P,1s to be the pivot column. — To determine the m;, we have 


IEEE S HE 
Thi TT» LUE 
orm; — 1,705 — 1,m; 52. The, — —zjy/m; are then 
Aye 14r Nd o dau che 


and 
^ — max X -— 7$ 


'The new constraint is then given by 


PT — ——ÀÉ I 
[s] [s] s 
D "A 2 /À 


—4 — Ee AT —19 — 2x5 d Z1 


'This equation is shown added to Tableau 9.2. By applying the elimi- 
nation transformation with the pivot element as shown, Tableau 9.3 is 
obtained. Since all the elements of column P, have been reduced to 
zeros, it can be eliminated from the tableau and column P; substituted 
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Tableau 9.3 


in its place. In fact, since vectors P4, P5, P;, Pj, P5, P5 will not change 
under the all-integer algorithm transformation, the tableau can be made 
quite concise as Tableau 9.4. Note that z, — zi is still in the basis and 
now hasa value of 4. Here there is only one row with a negative constant 
term. "The pivot column is P; since 


Ex QM i imd po noo 
—] -—1 
orm; € 10,m; € l. Hence ^; — 249,3 — 94, A — 83. The new con- 
straint 1s given by 
aet quc dos dis 
as shown in Tableau 9.4. "T'he new solution is shown in Tableau 9.5. 


Tableau 9.4 


Selecting row 3 as the row to generate the cutting constraint, we have 
P;asthepivot column and^ — 24. Thenew constraintis —1 2 —z; -- zs. 
The next solution is then given by Tableau 9.6. "With row 2 as the gener- 
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Tableau 9.5 


e 


ating row, we have Ps as the pivot column and ^ — 3. "The new con- 
straintis —1 — —zs-- zj. This yields the final Tableau 9.7 and the 
optimum solution. "The final solution is given by 


/ E /? [4 
212z2,—1 z227z2,—-0 z-72,-2 
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An example of the type of problem that can be solved by the integer- 
programming algorithm is the traveling-salesman problem as developed 
by Miller, Tucker, and Zemlin [74/]. 


A salesman is required to visit each of n cities, indexed by 1, . . . , n. He 
leaves from a '*'base city" indexed by 0, visits each of the n other cities exactly 
once, and returns to city 0. During his travels he must return to 0 exactly t 
times, including his final return (here t may be allowed to vary), and he must visit 
no more than p cities in one tour. (By a tour we mean a succession of visits to 
cities without stopping at city 0.) It is required to find such an itinerary which 
minimizes the total distance traveled by the salesman. 

Note that if t is fixed, then for the problem to have a solution we must have 
tp Z n. Forí — 1l, p Z n, we have the standard traveling-salesman problem. 

Letd;;(475j — 0,1, . . . , n) be the distance covered in traveling from city 
itocityj. The following integer-programming problem is shown to be a model of 
the problem: 

Minimize the linear form 


2 diti 


0i En 


over the set determined by the relations 


2 z4—1 jl, Qn 
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iz 

T7 
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where the z;; are nonnegative integers and the u, (1 — 1, . . . , n) are arbitrary 


real numbers and can be restricted to nonnegative integers. If t is fixed, we 
LO 


must add the restriction » gr 
ica 

From a theoretical point of view, the techniques of Gomory and 
others should enable us to solve any integer-programming problem. "The 
digital-computer codes based on these algorithms have, in most instances, 
proved unpredietable in their ability to guarantee convergence to the opti- 
mum. The success of such algorithms varies according to the problem 
and the rules used to develop the eutting planes. This is in sharp con- 
trast to the highly successful use of the simplex method to solve almost 
any standard (continuous) linear problem. "The reader is referred to 
Balinski [3a] and Trauth and Woolsey [946] for diseussions comparing the 
efficacy of different algorithms and related computational experience. 
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4. THE DECOMPOSITION OF LARGE-SCALE SYSTEMS 


Although it is theoretically possible to solve any given linear-programming 
model, the analyst is quickly made aware of certain limitations which 
restrict his endeavors. Chief among these limitations is the problem of 
dimensionality. Almost all diffieulties that arise in the development of a 
programming problem can be related to its size. "Thisis certainly true for 
such restrictive items as the cost of data gathering, matrix preparation, 
computing costs, and validity of the linear model. 

The development of procedures for the solution of large-scale systems 
is reviewed in Dantzig [27d] and Gomory [51/], and a number of proce- 
dures are given in Graves and Wolfe [51g]. Dantzig discusses techniques 
to reduce the computational requirements of large systems from two points 
of view: (1) by decreasing the number of iterations, and (2) by finding a 
compact form for the inverse and/or by taking advantage of any special 
structure of the system of equations. To cut down the number of itera- 
tions, variants of the simplex method have been proposed to replace the 
usual Phase I of the simplex method: method of leading variables, Beale 
[5]; composite simplex algorithm, Orchard-Hays [83] and Wolfe [1056]; 
and primal-dual algorithm, Dantzig, Ford, and Fulkerson [28]. 

Proposals for finding a compact form for the inverse or for taking 
advantage of special structures include the sparse-basis technique, 
Markowitz [74c]; block-triangular basis, Dantzig [20, 27/]; the solution of 
a dynamic Leontief model with substitution, Dantzig [27e]; large-scale 
economie systems, Rech [86a]; and for the important elass of block-angular 
systems and multistage systems of the staircase type we have the decom- 
position algorithm, Dantzig and Wolfe [320]; partition programming, 
Benders [65], Rosen [87c], and Ritter [87a]; pseudo-basic variable procedure, 
Beale [5c]; and the dualplex method, Gass [460]. In this section we shall 
describe only the decomposition algorithm of Dantzig and Wolfe which, 
when applied to block-angular systems, has an interesting and important 
economic interpretation in terms of decentralized planning; see Dantzig 
[16c] and Baumol and Fabian [3g]. 

For many problems the constraints consist of rather large inde- 
pendent subsets of equations which refer to the same time period or same 
production facility. "These subsets are usually tied together by a small set 
of equations. "These ''tie-in equations" might represent restrictions on 
the availability of basic resources, total demand of a product, or total 
budgetary constraint. In problems of this sort we have, in a sense, à 
number of separate linear-programming problems whose joint solution 
must satisfy a set of additional restrictions. If we boxed in the sets of 
constraints and corresponding part of the objective function, Fig. 9.3 
would result. 
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i 
FIGURE 9.3 


In Fig. 9.3 we have partitioned the original problem of AX - b, 
X » O, cX a minimum, into the decomposed program! of finding the 


veetors X, —- O (p —-0,1, .. . , k) which minimize 
k 
NONE (4.1) 
p-0 
subject to 
k 
» A,X, — bo (4.2) 
p-0 
BN, b. pueq1og. cec (4.3) 


where À, is an mo X n, matrix, B, is an m, X n, matrix, C, is an n,-com- 
ponent row vector, b, is an m,-component column vector, and X, is an 


k 
nj-variable column vector. Hence, the above problem has m — » m, 
p-0 


k 
constraints and m — 2: n, variables. In explicit matrix notation, (4.2) 
p-0 
and (4.3) are given by 
Ào A, A» US A; od bo 
Bi X; b; 
B; X» b; 
B,JiX,] |b, 


! Here we are adhering to the notation of Harvey [53a]. 
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To better understand what follows, let us look at the geometry of à 
very simple decomposed problem: minimize cjixii d- ciox;? subject to 
X, — (zi,21)) 2 0 and 
Z1; £t € AiX; X b, 


(4.4) 


du 215 
3zi d 4xas 


BiXiXb, 


The nonnegative solution set to (4.4) is shown in Fig. 9.4. The hatched 
area is the solution set to BiX, € b, and the shaded area is the solution 
space to the complete problem (4.4). "The extreme points of B,X, € bi 
are indicated by a square, while the extreme points of the complete prob- 
lem areindicated by a circle. Any solution to the complete problem must, 
of course, satisfy BjiX; € b,. But since the solution set of linear con- 
straintsis a convex polyhedron, any solution to B;X, € b; can be expressed 
as a convex combination of the extreme points of its solution set (we 
assume this convex set is bounded). Hence, if we knew all the extreme 
points of BiX; € bi; or could find the right ones, all we would need to do is 
to find the convex combination of these points which satishes A;X, X bo 
and optimizes the objective funetion. This is the essence of the Dantzig- 
Wolfe decomposition algorithm. In Fig. 9.4 we note that any circled 
point is also a convex combination of the squared points and that any point 
in the shaded area can be represented as à convex combination of the 
squared points. 

Extending this idea to the general problem of (4.1) to (4.3), we assume 
that for each p there is available the corresponding solution set S, to the 
constraints B;,X, — b,, X, 2 O. "Then the solution of the original prob- 
lem could be thought of as the selection of à convex combination of solu- 


X12 


FIGURE 9.4 
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tion points (i.e., extreme points) from each S, so as to satisfy the tie-in 
k 


k 

restrictions ba A,X, — bo and make » C,X,a minimum. 1f our origi- 
p-0 p-0 

nal problem is looked at in the above manner, we shall see that we would 

then be required to optimize a new problem with mo -F k constraints 

subject to the solution of k individual m, X n, subproblems instead of 


k 
one large problem with E mj constraints.! 
p-0 

The new problem to be considered is called the erxtremal problem or 
master problem and arises in the following fashion: Let the extreme points 
of each S, be denoted by Xj forj — 1,2, . . . , N,, where X,; are solution 
vectors for the pth subproblem and N;, is the total number of extreme 
points for S,. Any feasible solution X, to the pth subproblem ean be 
expressed as a convex combination of extreme points of S,, that is, 


N» 
X, -— 25 NX y; (4.4a) 
j-21 
where 
N, 
93 às; 1 and Ay; 2.0 (4.4 b) 


We are temporarily assuming that each S, is a bounded polyhedron. 
From (4.4a), substituting the expressions for X, into (4.1) and (4.2) 
we have, respectively, 
Ny, 


k 
CX » €: XuXs 
p-1 j-21 


and 
k 


N» 
AoX; — b A, ( b XajX.o; SR bo 
p-l1 gs 


Or 
kN, 
CX, Y bj Q,X (4.4c) 
mels3-l 
and 
kN, 
AX, d » 2) AX; (4.4d) 
plug 


Define the transformations 
P, — AQX, and j5 CX fixed ote ME (4.5) 


and substituting in (4.4c) and (4.4d) we have that problem (4.1) to (4.3) 
can now be written in the equivalent form of finding ,; 2 0, X, » O, 


! From the design of present computer procedures for solving linear-programming 
problems, we note that the number of constraints is usually the restrictive element. 
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which minimizes 


kN, 
CX, 4 x Y» T »jNpi (4.6) 
p-17j-1 
subject to 
kN 
AX, 4L y 23 DUX ER b, (4.7) 
Á pq 
Y NM-l ped 2 x d (4.8) 


This problem is the extremal problem. The first mo constraint rows are 
termed the transfer rows and the last k rows are the convexity rows. In 


(4.7) and (4.8) the columns pu that 1s, the columns of A, with k zeros 


) are eztremal columns, 
p 


where e, is the pth column of a unit matrix of order k. We should empha- 
size that (4.6) to (4.8) are just the representation of (4.1) to (4.3) in terms 
of all the extreme points of all the k subproblems. "The transformations 
P,; and f,; are just the representation of these extreme points in terms of 
their contribution to the tie-in equations and objective function, respec- 
tively. Given the optimum A,; to the extremal problem (4.6) to (4.8), 
E the corresponding optimum solution i in terms of each X, is given bx 


NES istante VM X; ; 7 


To illustrate the above let us reed thé extremal problem for 
problem (4.4). "The original problem in equation form, after the addition 
of slack variables xo:, 15 and z14, 18 to minimize 


P 
attached, are natural columns; and the columns ( a 


746r m. fs 


€11311 o. Cist? 


subject to 


ipe ee 4Z12 


—ac -- 22:1» -b- 21s 
3zii d 4xus en 


—1L-2 Xx 
with all z; 2 0. Here À, — (1), Ai — (1,4,0,0). 5-( d p 2 


2 
z— (12) bi z- bu C, z— (0), 5 E (€11,012,0,0). In Fig. 9.4 the 
extreme points of the solution set to B;X; € bi are denoted by Xi,X1, 
Xi,Xi. We note that we are now dealing with equations and the X;'s 


-t 
NL Rd 
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have four components corresponding to zi; zi», and the slack variables 
215, fj;4.] Following Eq. (4.5), we have 

P -AÀXa- (1400)X1n 

P; —- AiX5, — (1,4,0,0)X 5; 

P;; — AiX4 — (1,4,0,0)X 3; 

Pj —- AiXy, — (1,40)0)X 14 


Ju Cin (uno 00)Xu 
fi; — C€iXis — (01,01,0,0)Xi 
fis 2 CiXis 5 (01,015,0,0)X35 
fi 5 €iXu 7 (05050,0)Xi 
'The extremal problem, that is, the original problem expressed in terms of 


all the extreme points of the subproblem B;X,; € bi, is to find 34; 20 
which minimizes 


Zoo P fiii E figs faisais d fiia (4.6a) 

subject to 
£90 d- Pii d. Pisas d- Pii 4- Pi — 12 (4.7a) 
iuc — aco nc — 5na-—1 (4.8a) 


As mentioned above, this transformation yields a problem with only 
mo 4- k constraints, the mo transfer rows (4.7), and the set of k convexity 
constraints (4.8). However, the number of variables has been increased 
to the total of all extreme points of the convex polyhedra S,. "Thesaving 
element of the decomposition principle 1s that we need consider only a 
small number of this usually rather large total, and we need only the 
explicit representation of those to be considered. We shall see that the 
decomposition algorithm embodies the basic elements of the pricing 
vector and basis transformation of the revised simplex method. We 
next outline this algorithm and give a numerical example. 

We treat the extremal problem (4.6) to (4.8) as we would any other 
linear-programming problem except that we have to allow for the fact that 
we are unable to write out the matrix of the system corresponding to (4.7). 
However, let us assume that we know enough about the system so that we 
have an initial basic feasible solution to the extremal problem. — This first 
solution eould have been obtained using an artificial basis or, based on 
the structure of the subproblems, we were able to find extreme-point 
solutions to the subproblems which combined into a feasible solution to 

T To avoid notational confusion, the reader should bear in mind that X; is the 


variable vector with variables (components) zi, zi», zis, zi, and the extreme points Xi; 
are particular solution vectors with specified values of the variables. 
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the extremal problem (see example). This basis is of order ms 4- k and 
must contain at least one extremal column from each subproblem solution 
space S,, that Is, contain in total at least k extremal columns and mo 
additional extrema] or natural columns or both. Denote the associated - 
simplex multiplier vector! by IX — (2,2), where the m,-component row 
vector « is associated with the m» transfer rows of (4.7) and the k-com- 
ponent row vector x ls associated with the convexity rows (4.8). The 
components of « are called the transfer prices and will be denoted by 
v; and those of x by z,. For the vectors in the basis we must have 
f» — «P, 4 7,; that is, the corresponding c; — z; — 0. Asin the regu- 
lar simplex method, we next have to determine whether or not the initial 
feasible solution can be improved by introducing some new natural or 
extremal column. Pricing out an extremal column from some subproblem 
p, we have from (4.5) relative costs (that is, c; — 2j) 


fu - n (en) — fa — «Pu — es m CX nA m 
— (C, — xÀ,)X,; — v, Dos CMS (4.9) 
Pricing out the natural columns, we have, for each j, the relative costs 
Co — «Àj (4.10) 


where co; are the elements of C... Since the extremal problem is a mini- 
mization problem, if all the relative costs (4.9) and (4.10) are nonnegative 
for all p and j, then the simplex criterion tells us that we have an optimum, 
since (4.9) and (4.10) correspond to the set of c; — z; of the simplex 
algorithm, and for a minimization problem we are optimalifallz; — c; € 0 
orif allc; — 2; 2 0. "The costs (4.10) are readily calculated, as we know 
allthe Ay. However, in order to determine if any of the relative costs 

- (4.9) are negative, we need to solve p optimization problems which arise 
as follows. For any p we need to know whether or not for the set of 
extreme points X,; we have 

minimum [(C, — xA,)X, — 7,] « 0 pss at oes dn 1 UOTE 

onm 
But (4.11) states that for each p the quantity in brackets is minimized 
over those extreme points X,; which satisfy the constraints of the pth 
subproblem. If the minimum is nonnegative, then all extreme points of 
the pth subproblem price out optimally. 

Since the condition (4.11) is independent of the scalar z,, (4.11) 
reduces to solving for each p the linear-programming problem, which is 
to minimize 

(C, — zAj)X, (4.12a) 

! We assume we are employing the revised simplex method to perform the optimi- 
zation of the extremal problem; see Chap. 6. 
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subject to 
B,X, — b, (4.12b) 
X20 


We note that since the solution to (4.125) by the simplex procedure will 
generate only extreme-point solutions, then our solution to subproblem 
(4.12a and b) will also satisfy (4.11). We also note that each subproblem 
(4.12a and b) is equivalent to the given subproblem except that the 
corresponding cost coefficients have been reduced by zA,; the (C; — «A;) 
are called adjusted costs. We see that «A, is found by taking n, inner 
produets and can be readily ealeulated. We let X,, be an optimum 
solution to (4.12a and b) for a given p (X,, will be an extreme point if S; 
is bounded) and let the corresponding optimum value of the objective 
function be denoted by 


Z pa Fw (C, u nÀ,)Xy 


If Zj, — 7, « 0, then the new extremal column formed by the transfor- 
mation (4.5), that is, P,, — A,X,, with its associated cost fj, ^ CX, 
is a candidate to enter the basis. The new extreme point X,, 1s called 
a proposal vector, the transformation vector P, its transfer vector, and f, 
its transfer cost. 

Given the set of minimum oosts (4.9) and the costs (4.10), the simplex 
method tells us to take as the next column to enter the extremal-problem 
basis the one corresponding to the min (c; — zj), that 1s, 

min[min(Z;, — 7,), min(co; — Àj] (4.13) 

p 3 
where p ranges over the set of subproblems and j ranges over the nonbasic 
naturalcolumns. If (4.13) is nonnegative, then the current basic feasible 
solution of the extremal problem yields an optimum to the original prob- 
lem. If (4.13) 1s negative, we select the corresponding extremal or natural 
column to enter the basis. For this latter case, we determine the pivot 
row as usual, determine the new II, and repeat the procedure until an 


optimum is reached. "The optimal solution to the original problem is 
given by 


[Xo, X, x Y NX (p z— 1; 2, gu gon k)] (4.14) 
7 


where the summation is taken over those extreme points of the pth sub- 
problem which are in the optimal solution to the extremal problem. 

Up to this point we have assumed that the polyhedra S, are bounded. 
For the general situation let us assume that this is not the case. Then in 
optimizing one of the subproblems (4.12a and b), we could obtain an 
optimum solution with an unbounded value Z,, of the objective function; 
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see Sec. 1, Chap. 4. For this case the optimum solution to the subproblem 
will yield a corresponding transfer vector P,, as a candidate for the 
extremal-problem basis, but we cannot restriet the associated extremal- 
problem variable A;, to be bounded by the convexity equation; that is, 
^,, ls not bounded above. "The reason for this is as follows: In the 
simplex method an unbounded solution is represented by a basic feasible 
solution plus à nonnegative sum of vectors (see Exercise 19). For the 
standard linear-programming problem this latter sum is a solution to 
the homogeneous equations AX — O and the constraints X » O. For 
the decomposition problem, we would have a nonnegative solution to 
the homogeneous equations B,X, — O. Welet X,, denote this solution, 
B,X,, — O, and treat it as any other proposal vector (note that X,, 
is not an extreme-point solution to the subproblem). AsB,(^,,X,,) — O 
for all 444 7 0, we have that a nonnegative sum of X,, with à convex 
combination of extreme points of S, can be found which yields a vector 
satisfying the constraints B,X, — b,. Whenever we encounter an 
unbounded solution X,, in any subproblem p, the corresponding con- 
vexity row 1s rewritten so as not to include the appropriate A5,, that is, 


From a computational point of view we should note the following. 
Each time it 1s necessary to reoptimize a subproblem for a new objective 
funetion, the previous optimal feasible solution should be used as the 
first feasible solution to the new problem. In the selection of the new 
nonbasic variable to enter the basis, we need not obtain the true minimum 
(4.13), but can select any vector which would yield an improvement. 
Computational experience has shown that it appears to be best if a num- 
ber of vectors which would individually yield an improvement, possibly 
some from each nonoptimal subproblem, are simultaneously considered 
as transfer vectors in the optimization of the extremal problem. That is, 
we do not restrict an iteration of the extremal problem to just a preselected 
basis change, but allow the old basic vectors and a number of new transfer 
vectors to be candidates in the formation of the new basis. Large 
systems of the order of 5,900 rows and 8,000 columns, with 175 tie-in 
restrictions and 10 subproblems, have been solved in 18 hours on an IBM 
7094 computer (Hellerman [54a]). "We should note that the applieation 
of the decomposition algorithm is not always successful in that convergence 
to the optimum might require an inordinate number of iterations. Even 
successful applications, as noted above, can use a great deal of costly 
computer time. The field of large-scale linear-programming systems will 
always be a challenging one and still requires much experimentation and 
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new developments. For a general discussion on approaches to solving 
large-scale systems, see Gomory [51/]. 

It should be stressed that although the decomposition algorithm was 
developed for structured problems, any linear-programming problem can 
be decomposed. For example, if a computer code is limited to the 
handling of a problem with 500 constraints, an appropriately designed 
decomposition eode could subdivide a larger problem into two parts 
which could be readily solved. Also problems which involve transporta- 
tion or network problems as the subproblems, e.g., the multicommodity 
problem, ean use the special algorithms to solve the subproblems and 
the revised simplex method to solve the extremal problem. The fact 
that any linear-programming problem can be decomposed in a manner 
that best suits its special form is the basis of an application of the decom- 
position principle to the transportation problem (Williams [1004]). 

Extensions of the decomposition principle include (1) the generalized- 
programming problem of Wolfe (Dantzig [16c]), where each column 
vector of a linear-programming problem 1s allowed to be selected from a 
convex set (see Gilmore and Gomory [49a] for an applieation of this 
technique), and (2) the dual solution of the extremal problem and para- 
metric decomposition procedures of Abadie and Williams [1c]. As the 
solution to the extremal problem can yield a nonbasic optimal solution to 
the original problem, Abadie [1a] modifies the computational procedure 
so that the current solution to the extremal problem is always a basic one 
for the original problem. 


Example. Minimize 
—4Z1 Fr Z1 — 3251 — 2225 
subject to 


Xii — 3zis 
Ti2 


3zii 4d- 221» 


(4.15) 


—Z1 -k3zis 


and 
Xi 2 0 


Rewriting (4.15) in terms of equations with slack variables xo;,zos, 
113,014,1223,224, We have 
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To TEn — 3zi$ —2ux» d- X339 —- 6 
02 SEDE: —-Om1 op zd 
3r -cF 2zis 4 xus —12 
E2010 31:15 -- X14 - Eon 
3Xxsi -- X99 d- X33 


Eo d 
m 
t2 


X21 d- 222» rx 


C, — (0,0) C, *x (T0) C; c (20) 


Treating the subproblems as inequalities with nonnegative slack 
vectors, the solution spaces S; and S; to the subproblems are given 
in Fig. 9.5a and b. We have indicated all the feasible extreme-point 
solutions. 

To find a first feasible solution to the extremal problem,! 
we consider the extreme-point solution to the first subproblem 
Xi; — (0,0,12,6) and the extreme-point solution of the second sub- 
problem X»; — (0,0,12,8), where the last two components of X, are 
the slack variables zi and zi4, and the last two components of X;i 
are the slack variables zs; and x». Then P, — M" Pa i) 
fui — O0, fa1 7 0, and the extremal problem is given by ^ 


1 0 0 0 6 
0 1 0 0 4 
0 Cer " X02 c 1 ud 0 Àoi — 1 
0 0 0 1 1 


with the first feasible solution of zo1 — 06, xo — 4, Ài1 — 1, às — 1, 
and as all co; and f,; are zero, the value of the objective function is 
zero. Notethat II — (0,0,0,0). For this pricing vector Hi, we need 
to see if any other extreme points from the subproblems can improve 
the value of the objective function. Since both « — 0 and x - 0, 
the optimization of the subproblems just involves the original data 
of the subproblems; i.e., the objective functions of the subproblems 


! See Exercise 24 for a general procedure for finding a first feasible solution. 
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For BjX, — b, 


X12) 


FIGURE 9.5a 


For B2X; — b; 


»-— X21 


FIGURE 9.55 


are not changed. We then have to find the extreme-point solutions 
of the following problems: 

Minimize 

—SEap- 342 


subject to 


32131 Es Do 4 113 NIE (4.17) 
—Zn-r v1 Tira 6 
Xi E 0 


and minimize 
—8Xs — 2433 


subject to 


9Zai- X22 Fo X2& — 12 (4.18) 
X91 c- 2x2» cT: 8 
Xi 2 0 


We would ordinarily use the simplex method to solve (4.17) and 
(4.18), but since we have the complete set of extreme-point solutions 
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in Fig. 9.5a and b, respectively, we shall just evaluate each extreme 
point in the corresponding objective function and select the minimum. 
For the first subproblem we have X, — (4,0,0,10), Zi — —4, and 
for the second subproblem we have X»; — (165,124,0,0), Zj, — —72£. 
We note that all the vectors of A, are in the current extremal basis 
and that min[ZisZ»;] — — 72$. "Therefore, by (4.13) we select 
X»; to enter the basis of the extremal problem. [Its transfer vector 


—4 : e 
P,- o and its transfer cost fi» — — 725. "The extremal prob- 
ve 


lem in tableau form is now 


[o er 


'To improve the solution, we introduce ^s» into the basis, and to 
preserve feasibility, we eliminate ros. "This 1s shown in the next 
tableau. 


We drop the column associated with xo; if 1t 1s needed for a future 
basis, it will be brought back to the extremal problem at that time.! 
Our extremal solution is xoi — "46, Xi — l, Aa — 46, Ass — 96, 
Z — —8. 'The corresponding feasible solution. to the original 
problem (4.16) is X, — (726,0), Xi — AuXi - (0,0,12,6), and 


X5 ks AeiXoi *- AooXo» sd 16 (0,0,12,8) T 96 165,125,0,0) 
- 16(16,12,48,92) 
1 For more involved problems, it appears as if it pays to keep the past extreme 


points as candidates to reenter the extremal basis as long as it is computationally 
convenient. 
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with Z 2 —8. "The corresponding EH -— (0,—2,0,0); that is, 
z — (0,—2), x — (0,0). The reader should recall that II — IB 
where f is the vector of cost coefficients of the vectors in the current 
basis B. We leave it to the reader to caleulate B^!. Using this 
pricing vector, we repeat the process to see if à new extreme-point 
solution. from the subproblems ean improve the solution. From 
(4.12a) the subproblems now have the respective objective functions 
of 
minimize —a1i c 31» and minimize 3x»; — 45» 


For subproblem 1, Xi; yields the optimum with Zi» — —4; for sub- 

problem 2, X;, yields the optimum with Z;; — —16. Since 

min (co; — xA») — 0, we then select X»; to be the proposal vector 
4 


with the transfer vector of P»; — ( 


e and transfer cost of f»; — —8. 


The complete new tableau of the extremal problem is now 


[2 er 


Introducing ^s» as before, we have 


The reader will note that the extremal column associated with P;; 
has now been transformed, as would be the case in the revised simplex 
method; i.e., it has been multiplied by the inverse of the current 
basis of the extremal problem. In practice, we would not set up 
the first complete tableau, but just form the transformed extremal 


column and introduce the new variable. We next introduce A5; 
and eliminate ^5, to obtain 
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For this solution H — (0,—44,0, —722); that is, « — (0,—42), 
& — (0,— 724), and the new objective functions for the subproblems 
are 


minimize —;; 4- !lez;» and minimize —94r5, — 18475, 


For subproblem 1 the minimum 1s assumed for Xi» — (4,0,0,10) with 


Zi» — —4 and fi — —4; for subproblem 2 the minimum is 
assumed for both X», - (16,12£,0,00 and X; — (0,4,0,0) with 
Za» — Za; — — 724. Also co — mÀo» — 34 » 0. We then select 
X1» as the proposal vector since Zi» — 71 ^» —4 —0 — —4 « O0 and 
Za) — T3 — Za — $5 — — 724 -- 724 — 0. (Note that the extremal 
column corresponding to X»; is a current basis vector of the extremal 
problem; hence Z;; — 354 — 0.) Pi; — (5) and fi; — — 4. 


ho 
NO [NO [MCN So 


-NM 


EQ 


-N 


E 
[E 
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'The new H. — (0,—44,—4,— 724), and the reader can verify that for 
this pricing vector no improvement can be obtained. 'The optimum 
solution to the original problem is 


X, (262,0) X,-— (4,0,0, 10) 
X; — 54(19$,126,0,0) J- 24(0,4,8,0) — 14(16,20,16,0) 


I 


with —1167 the minimum value of the objective function. 


5. BOUNDED-VARIABLE PROBLEMS 


An important class of linear-programming problems is one in which all or 
some of the variables are bounded from above. Mathematically we have 
the following problem: 


Minimize 

zc cX (D. D) 
subject to 

AX -—b (5.2) 

XO ESI 

z; € uj for all or some 7 (5.4) 


We have, of course, each u; 7» 0. To generalize the discussion, we assume 
that all x; are bounded above as we can always bound a variable by a 
suitably large number. The given upper-bound restrictions (or secondary 
constraints) can represent capacity limitations, production requirements, 
or resource limitations, and arise naturally in many applications. 

One way of solving this type of problem would be to replace the set 
of z; € u; by an equivalent set of equalities In nónnegative- vüriables 
—Kusmg' sláck variables). This, however, greatly increases the Tümber of. 
constraints to be handled. A more efficient technique has been developed 
-««Oharnes and Lemke [14] and Dantzig [24]) which entails only a few 
modifications of the standard simplex procedure and enables one to solve 
the problem without explicit representation of the upper-bound con- 
straints. (The reader will note that lower-bound conditions offer no 
difficulty; see footnote on page 248.) 

The justifieation of the special procedure for solving the bounded- 
variable problem ean be seen from the following (Dantzig [10c] and 
Simonnard [905]). In the general theory of the simplex method we deal 
with basie feasible solutions which are obtained by arbitrarily setting 
n — m nonbasic variables equal to zero and solving the resulting m X m 
square system, assuming we have an admissible basis. For the bounded- 
variable problem (5.1) to (5.4), in order to employ a similar technique, 
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we must allow the nonbasic variables to take on values other than zero. 
However, the situation is simplified, as we can restrict the value of a 
nonbasie variable to be either zero or the corresponding upper bound. 
-For-this-problem, we then define an extended basic feasible solution to be 
one for which » — m variables have been set equal to either their lower- 

-erupper bounds (zero or uj), the matrix B of the associated square system 
1s.a. basis, and the solution to the square system yields values of the basie 
variables which satisfy the upper- and lower-bound constraints (5.3) and 
(5.4). We then have the following theorem. 


Theorem 1. 4» ertended basic feasible solution is optimal 1f the 2; — cj of 
the nonbasic variables satisfy the conditions 


2;—c; X0 ifz;-0 (5:5) 
and 
f cct m —0 if X;-—UWuj (5.6) 


Proof. 'Theorem 1 can be proved by noting the following. For B, the 
basis of the extended basic feasible solution, let & — (mi . . . 74) 
be the associated simplex multipliers (see discussion in. Chap. 6). 
Then multiplying (5.2) by « and (5.1) by —1 and adding, we have 


zÀX —cX —b—2 
Or 
z — nb — (xÀ — c)X 


For B, we have2 — z — » (2; — cj), where 2 — «b is the value of 
J«B 

the objective function for the basic variables.! "Thus 

min z — min [2 — Y (z; — cj); (5.4) 

j«B 

If conditions (5.5) and (5.6) hold, then any changes in the nonbasic 
variables will increase the total value of the objective function; thus 
(5.7) is at its minimum. 


If the conditions of Theorem 1 do not hold, we can change the current 
solution and determine a new, improved one. An analysis of the com- 
putational considerations enables us to apply the simplex algorithm to 
the primary constraints (5.2) and (5.3), but modified to ensure that the 
secondary constraints (5.4) are not violated. Thus, the size of the basis 
will be only of dimensions m X m. Compared with solving the complete 


! 'The notation » means that the sum is taken for the set of indices j of those 
j«B 
vectors not in the basis B. 
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bounded-variable problem by the simplex method, the modified procedure 
represents quite a computationalsaving. By using the bounded-variable 
algorithm, we do not reduce the number of iterations, but decrease the 
time per iteration and are able to solve rather large bounded-variable 
problems. A discussion of the bounded-variable algorithm follows 
(Garvin [45a] and Simonnard [905]). 

Assume that we have an extended basic feasible solution with basis 
B and that conditions (5.5) or (5.6) do not hold for some nonbasie variable 
zx, lf we want to increase z;, then either some basic variable will reach 
its upper or lower bound or z; will attain its upper bound. Similarly, if 
we want to decrease r,, then either some basic variable will reach its 
upper or lower bound or z; will attain its lower bound. If r, reaches one 
of its bounds first, then we will not have to change the current basis B. 
If some basic variable, say z;, attains one of its bounds first, then z, will 
be replaced by z,in thebasis. We next have to determine the appropriate 
value of z, for which all the constraints of the problemi will remain satisfied. 
An improved solution will be obtained as we selected an z; for which (5.5) 
or (5.6) 1s not satisfied. 

Case 1l. Determination of a new basis 1f z, — O0. 

For the given solution we have z; — 0 and z; — c; » 0. In terms of 
the simplex transformations for the current basis (see Tableau 4.1, Chap. 
4), each variable z; in the basis is given by 

4. 99 qug —— 32 XU; — XidkXk (5.8) 

jeU 
where U is the set of indices for those nonbasic variablesxhich equal their 


upper bounds. ! Letüng b, $9 — 2j Tiu; (. 8) becomes 
jeU 


gy; b; — Sach for all Z eB 
For those ? for which z;, 7 0, we see that an increase in z; from its current 
value of zero will decrease the corresponding r, Thus we must restrict 


any increase to z, to a value which does not allow any of the eürrent t basic 
variables to become negative. We must have for z; » 0, Iz 


b; — Xa. et) 
Or 
bi 
Za 


or 


b cus 
XQ € -— x mimm —- 
vise X pk hem 3A (5.9) 


! 'The notation 2 means that the sám is taken for all i in the set of indices U: 
j«U 
eB represents the set of indices of the basis vectors. 
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Similarly, for zi. « 0, an increase in z; will also increase the corresponding 
z, But each z;is bounded above. "Thus we must have for zi; « 0, 


b; — Xu es ug 


u;—b 
Xk . D 1 
—Xik 
Wsteeb COMI ces D. 
zr, X —— —! — min ——— (5.10) 
— Xak zikc0 —— Vik 


Finally, we also must have that 
T SS Uk (5:10) 


Taking (5.9) to (5.11) together, the maximum increase in z, is given by 


D UE u; — bi 
max z, — min]| — for za 7 0, ———— forza « 0, wu, 
ieB Vk — Jk 
sol Dos dio b 
maux eon em c 
X»k — Xgk 


If max z, is limited by (5.9), then x, replaces z, in the basis; if it is limited 
by (5.10), then z, replaces zx, in the basis; if 1t is limited by (5.11), then 
x, stays nonbasie but its value is now &,. In the first two situations the 
basis is changed using rj, or r,, as the corresponding pivot. We note 


ina the pivot could be negative. 


Case 2. Determination of a new basis 1f x, — wu, and z, — c, « O. 
For this ease we have 


Bb m— onec » Xu; — Xiktk (5212) 
je 
jzék 
. I: S em T B 
Letting 6; 2 zi; — » zijuj (5.12) becomes z; — b; — züz, for all ? e B. 
j«U 
Jzk 
For zi, » 0, a decrease in z, will increase z;, while for xj « 0, a decrease 
in z, will decrease z;; We must ensure that a decrease in z, will not allow 
an z; in the basis to go below zero or above its upper bound. Thus we 


must have 


yz; b, — xaxa € wi loro 09 
or E A 
PN Tuc J 
29». z- b, — up -— ine uu ag (5915) 
X pk zk»0 — Vid 
and , 
dA b; — UVueUk 2 0 for Jk — 0 
Or 
b. " 
X& zz o— — max-— (5.14) 


qk zikc0 Tik 
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and finally, 
z, 20 (5.15) 


Combining (5.13) to (5.15), we have that the minimum value of z; is 
given by J ] 
' b-— uj Ub. 
min z, — max E —L, o| 


Xpk Xok 


If min z; is limited by (5.13), then z, replaces z, in the basis; if it is limited 
by (5.14), then z, replaces z, in the basis; if it is limited by (5.15), the old 
basis stays the same and z, is a nonbasic variable equal to zero. 

In either case, we obtain the largest decrease in the value of the 
objective function by making the variable z, equal to the value which 
constrains it; that is, we increase zr, as much as possible in Case 1 and 
decrease it as much as possible in Case 2. After an appropriate change 
in the variable z,, the above procedure is repeated until the conditions of 
Theorem 1 hold. Since all z; € w; the given problem has a finite mini- 
mum. The development of a computational tableau and related rule 
for the selection of the variable z, to enter the basis is left to the reader in 
Exercise 206. 

Other approaches to the bounded-variable problem employing dual 
relationships are given in Eisemann [41c], Gass [465], and Wagner [994]. 
Special algorithms exist for solving the capacitated-transportation prob- 
lem in which each shipment z; X wi; (Ford and Fulkerson [44a] and 
Garvin [45a]). A more general approach to upper bounds 1s given by the 
algorithms of Dantzig and Van Slyke [32a]. "They show that a linear- 
programming problem with m 4- | rows, such that each variable has at 
most one nonzero coefficient in the last | equations, can be solved using a 
basis of order m or 2m, depending on the algorithm. — The last / equations 
are general sums of variables, with each variable appearing in at most 
one of these equations. Problems of this sort include the transportation 
problem, bounded-variable problems, and distribution problems. 

All the above procedures can be looked at as special approaches 
which enable us to solve large-scale linear-programming problems. These 
algorithms, along with the decomposition and related techniques for 
handling the distinctive structures which usually arise in large-scale pro- 
gramming models, have added immeasurably to the utility of such models. 
However, as we now see the need to solve extremely large problems (of 
the order of many hundred thousands of constraints and millions of varia- 
bles) and as this need is now coupled with the ineredible computing power 
of today's electronic computers, the development of further advances in 
this area should be a prime field of research. It should be noted that the 
mechanies of the bounded-variable algorithm were discovered by the 


—mpphiestton-of the standard simplex 1 nom e Ene the enlarged e DOR 


DUERME MTESRHEEN LII OPPOSITO SAT NIRE 
— 


Sec. 6] ADDITIONAL COMPUTATIONAL TECHNIQUES 201 


"This fact, plus similar experiences, leads us to suggest that, whenever one 
Is faced with a new linear-programmüng model, one should constrüet" 
numerical examples, arrange them in the simplex tableau, and solve the 
—problems. An investigation of the computational tableaus might reveal 
hidden eharacteristies of the equations which could possibly lead to a 


special and more efficient computational method. 


6. DIGITAL-COMPUTER CODES 


The first successful solution of a linear-programming problem on a high- 
speed electronic digital computer occurred in January, 1952, on the 
National Bureau of Standards computer, the SEAC. The computational 
method used was the original simplex procedure, and the application was 
an Air Force programming problem dealing with the deployment and 
support of aircraft to meet stipulated requirements. Since that time, 
the simplex algorithm, or variations of this procedure, has been coded for 
practically all general-purpose electronic computers. 

For past editions of this book, we gathered from the computer manu- 
faeturers information concerning linear-programming codes for their com- 
puters. Because of the great variety of computers available today, it is 
rather difficult to maintain such a listing. For the current status of 
codes for any particular computer, the reader should inquire of the 
manufacturer. 

'The reader interested in computational characteristies of the simplex 
method 1s referred to Cutler and Wolfe [165]. "This study summarizes 
the results of a set of computations involving the solution of nine linear- 
programming problems using 30 variations of the simplex method. The 
statistics collected allow à comparison of most of the variations pro- 
posed in recent years and indicate the important features in the efficiency 
of linear-programming codes. 


EXERCISES 


1. Selecting vectors P;, P,, P; as an initial basis to the following equations, deter- 
mine the corresponding solution and, if necessary, transform the system to a form 
suitable for Phase I computation: 

? 


X; — 92» d Ax; «E r4. d r9 


pi bog c Gi. o daa vs gum 2 
—g4-F zs» — 3g; — 2z4, 4- 035 — 5 
7822200) 


2. Solve the following problem by means of the dual simplex method: Maximize 


xi m [x2 Eu 223 d za4 x 6x5 
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subject to 


QOyi — X.-F X3 — 274 

2 esede cupEti/Es Md 

—m$ -F 35$ — 324 Ei dg 
Tj 


Il 


Es 


0 


[Chap. 9 


3. Solvethe following problems in terms of integers, using the all-integer algorithm. 


a. Minimize 


zi-F 82» 
subject to 
22; 4-322 2 6 
— 3x1 c re s 1 
QE ms SS 
deem 3r» «x 1 
EE) 


b. Minimize 
Ti 
subject to 


3ri-l 22$ — 12 


—2zi| 4- 60x; — 12 
X2 « 
252-280 


c. Minimize 


€$1i-- 329 d- zs 
subject to 
zi d 225 d4- 3x3 2. 7 
—235;-F X.-LF zs € L8 
2; — 235 3-23 €. 5 
3xi d rs d Ux. — 14 
muss d) 


4. Solve a and b of Exercise 3 by graphing the constraints. 
b. Determine the noninteger solution to the example in Sec. 3. 


6. Solve the following problem using the decomposition algorithm: Minimize 


2xiài-— 3x» JF az4 — 2z4 4- 60r4 — 4x, 
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subject to 
$1 d 2x5 -F 323 4- 4x, — 5z4 — 2x, — 13 


m1 20-1. 55 — A 
3zi E 2r» 1 223 -— 


v pau 
2x, — zs 32, — 21 
Ur 0 


7. Phase I procedure for the dual-simplex algorithm (Vajda [97a] and Beale [5]). 
For the dual simplex algorithm let B be a basis such that B-!b has some negative 
components (not a feasible solution) with some associated 2; — c; 7 0 (the optimality 
conditions are not allsatisfied). Let |J] be the set of indices for the nonbasie vectors 


for which z; — c; » 0. Add the equation 2» z; 4 ro — M, where M 1s à very large 
jeJf 

positive number and z; 7 0. Let z, — c; — max z; — c; and in the new equation, 

E 

pivot zr, into the basis, which is now an (m -- 1) X (m -- 1) matrix. This procedure 

makes all the transformed 2; — c; X 0, and the values of the basic variables are 

functions of M. From this point on, show how the problem ean be solved in a manner 

similar to the Phase I procedure of the standard simplex method. What happens 

when z, goes into the basis? What if zo is not in the optimal feasible basis of this 

new problem? This procedure is the analogous dual-problem approach to the one 

artificial-vector tecbnique of Sec. 1, Chap. 9. 

8. Describe geometrically the phenomenon of cycling in the dual simplex method. 

9. Solve the dual of Example 2 of Chap. 7. 

10. Transformation of problems into integer problems (Dantzig [16c]). 

a. The fixed-charge problem has as constraints AX — b, X — O and the objective 
function cX to be minimized, where c; — djz; -- kjif z; » 0, and c; 2 Oif z; 2 0;that 
is, the k; are fixed, one-time charges applied to the cost if the activity r; is used at à 
positive level. "Transform this problem into à mixed-integer problem, where the zr; 
are not restricted to be integers, but special integer variables are needed to express 
the fixed-charge requirements. HiNT: Assume each z; has a known, finite upper bound. 

b. Show how any integer problem can be transformed to one in which all variables 
are restricted to be either 0 or 1. 

c. Show how to express the condition that z; can equal bi, b», or 5;, using integer 
variables. 

d. Using a variable restricted to be either 0 or 1, express the nonlinear condition 
X12» — O0 by two constraints which involve the nonnegative variables z; and zs by 
themselves. HiNT: Ássume that x; and z; have known, finite upper bounds. 

11. Formulate the following delivery problem as an integer-programming problem 
(Balinski and Quandt (3c]). A delivery firm has three large boxes to deliver and six 
different trucks which ean carry loads of varyingsizes. "Truck 1 can carry only order 1, 
truck 2 must carry orders 1 and 3 together, truck 3 must carry orders 1 and 2 together, 
truck 4 must carry all the orders at the same time, truck 5 can carry only order 2 and 
truck 6 can carry only order 3. Each truck can make one trip a day and all deliveries 
must be made on the same day. The cost of each truckload combination is cj. Are 
there feasible solutions to this problem? 

12. Solve the following continuous linear-programming problem geometrically 
(Trauth and Woolsey [945]). Isthe optimum solution rounded to the nearest integer 
a feasible solution to the problem? — What is the integer optimum solution? 
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Maximize 
qM 
subject to 


—9z; 4 10zs 


I^ 


B C OX» 


283zi — 22r» 


I^ 


24 
0 


IV. IA 


Tj 


13. The knapsack problem (Dantzig [16c]). Formulate the following problem as 
an integer linear-programming problem. A hiker has a choice of carrying a com- 
bination of n objects subject to a total weight restriction. | Each object has a certain 
value c; to the hiker and a weight aj. If bisthetotal weight he can carry, how should 
he determine the combination of objeets which maximizes the total value? How 
would you solve the continuous-variable version of this problem? 

14. What assumptions are required to prove the finiteness of the decomposition 
algorithm? 

165. Lower bound for the objective function in the decomposition algorithm 
(Dantzig [16c]). Denote the objective function (4.6) of the extremal problem by Z 
and let Z, be the current value of the objective function. Show that for A; — (0) a 
lower bound for Z is given by 


k 
min Z 2 Ze T 2 (ra — f) 
p-l1 


HINT: See Exercise 4 in Chap. 4. Multiply Eqs. (4.7) and (4.8) by « and z, respec- 
tively, and subtract from Z. 

16. Develop a decomposition algorithm for the transportation problem. 

17. Discuss the decomposition of a bounded-variable problem, where the full 
set of bounding constraints represents one subproblem (Hadley [524d]). 

18. Formulate the multicommodity problem as a block-angular system and dis- 
cuss how it should be decomposed. 


19. Prove that if an unbounded feasible solution exists for 
BX-b 
X0 
then we can find a family of feasible solutions which satisfies the homogeneous equations 
BX-0O 
X20 


HINT: See Sec. 1, Chap. 4. 


20. Solvethe following problem using the decomposition algorithm (Harvey [53a]). 
Minimize 


—Ó6zos — 4ro —bzi — Bri; —'[x»i — "l3s» 
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subject to 


Xii 


S AID 


UTIMS IS QoS 


— ae da 


TX3i CRO X22 de oTf33 
—2z» -d- Xa -F x24 


Answer: Minimum — —198, 

6 2 2 
0 4 4 
X, — (0,0) X0 0 41 0b 2 s 
102 0 0 

0 254 8 

0 504 16 

X; -— l$5 25 "E 2465 de P 1 

0 0 0 


21. Solve the following problem by the direct simplex method and the decom- 
position algorithm (Müller-Merbach [775]). 
Minimize 


—8zu x 3x12 - Sri s 6z»» 


subject to 


Azxii 4- OXis d- X21 d- 3z29 Ed 16 
4rxi — — i2 d 3z5 2 
£31 241» «8 
Orr cj 12 SS NU 
2x51 J- 333» X 9 
Aoi 4 222 x 12 
Tij zs 0 
Answer: Minimum — —8774$, 
d1 — l?7$o — zu — ?900 — zai — ?]Áo —— zm — $6 


22. For problem (4.4), compute the numerical values of the X;j. Determine the 
values of the P;; and f;; and solve the related extremal problem (4.6a) to (c) by the 
standard simplex method; then solve problem (4.4) by the decomposition algorithm. 
q;etoe13 - —2, 612 24. 

23. Solve the example (4.15) by the regular simplex method and the revised 
simplex method. 
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24. Develop an artificial-basis procedure for determining a first feasible solution 
to the extremal problem. HiwT: Use positive and negative unit vectors which corre- 
spond, respectively, to the positive and negative elements of by. Each vector has a 
nonnegative artificial variable. Employ an additional constraint which forces the 
sum of the artificial variables to zero if a feasible solution exists; i.e., Phase I corre- 
sponds to minimizing a variable which represents the sum of the artificial variables 
(Dantzig [16c]). 

25. Solve the following problem by the decomposition algorithm using the first 
constraint for the extremal problem and the last two constraints as a single subproblem. 


Maximize 
€i -LF 22» 
subject to 
zi-d- x2 238 
—2xi-- x «2 
11— z;Xl 
BOE) 


26. For the bounded-variable problem, develop a rule for selection of the variable 
z, to enter the basis and a computational tableau. 

27. Solve the following problems by the bounded-variable algorithm. 

a. Maximize 


210-59 
subject to 


2xi-d- z» € 18 


231—225 € 6 
—3zi-F6z; «€ 9 
Zi Es 
oe 
quoc 


b. Maximize z; — zs subject to the constraints in a. 

28. Make the necessary changes to the bounded-variable algorithm if some 2s 
are not bounded above. 

29. Modify the bounded-variable algorithm and the transportation-problem 
algorithm to solve the capacitated-transportation problem. 


PART THREE / APPLICATIONS 


CHAPTER 10 / THE TRANSPORTATION 
PROBLEM 


One of the earliest and most fruitful applications of linear-programming 
techniques has been the formulation and solution of the transportation 
problem as a linear-programming problem. The basie transportation 
problem was originally stated by Hitchcock [57] and later discussed in 
detail by Koopmans [66]. "The linear-programming formulation and the 
associated systematic method of solution were first given by Dantzig [18]. 
The computational procedure is an adaptation of the simplex method 
applied to the system of equations of the associated linear-programming 
problem. We shall discuss but not develop this special procedure for 
solving the transportation problem. The reader is referred to Dantzig 
[18] for the complete details of the method. — Àn alternate procedure for 
solving this problem 1s given in Ford and Fulkerson [43]. 

As the transportation problem 1s of great interest, we have included a 
number of basie theorems concerning the equations of its corresponding 
mathematical system. | So as not to digress from the main task of formu- 
lating and solving the problem, the proof of a few theorems has been left 
tothereader. As preliminary reading it 1s suggested that the reader refer 
to the applications described in Sec. 2 of Chap. 1. 


1. THE GENERAL TRANSPORTATION PROBLEM 


A homogeneous product 1s to be shipped in the amounts a4, d», . . . , Gy, 
respectively, from each of m shipping origins and received in amounts 
b, 05, ... . , b,, respectively, by each of n shipping destinations. "The 


eost of shipping a unit amount from the ?th origin to the jth destination 1s 
c; and is known for all combinations (7,7). The problem is to determine 
the amounts z;; to be shipped over all routes (7,7) so as to minimize the 
total cost of transportation. 

To develop the constraints of the problem, we set up Tableau 10.1. 
'The amount shipped from origin 7 to destination j 1s z;;; the total shipped 
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Tableau 10.1 


Destinations 


from origin £ is a; 2 0, and the total received by destination 7 is b; 7 0. 
Here we temporarily impose the restriction that the total amount e 
is equal to the total amount received; that is, » gi m » b; — A. The 


total cost of shipping r;; units ls cjz;. incea Gaehdivd sbinudouk has no 
valid interpretation for the problem as stated, we restrict each z;; 7 O. 
From the tableau we have the mathematical statement of the transporta- 
tion problem: Find values for the variables z;; which minimize the total 
cost 


» [m (X) 


DECR, Cu NN AIT (1:2) 
j- 
» EL bj 6 ES L 2. 2e, ft (1.3) 
and 
t5 20 (1.4) 


Equations (1.2) represent the row sums of Tableau 10.1 and (1.3) the 
column sums. In order for Eqs. (1.2) and (1.3) to be consistent, we must 
have the sum of Eqs. (1.2) equal to the sum of Eqs. (1.3); that is, 


Ls 


X Xw-XX«- DE Qe 


1-17j-1 j-1i-1 
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We note that the system of Eqs. (1.1) to (1.4) is a linear-programming 
problem with m 4- n equations in mn variables. 


Theorem 1. Ze transportation problem has a feasible solution. 


Proof. Since » des Y b; — A, we have the feasible solution z;; — ajb;/ A 
1-1 j-1 


for all (2,52). Each z;; 2 0, and (1.2) is satisfied, since 


n n 1 
. Nea a,b, M j-1 T: 
Xj — A -— Ei Qi 
j j—-1 


j-1 UE 


and (1.3) is satisfied, since 


ge um m 


1-1 1-1 


Form — 3 and n — 5,let us write out the equations that correspond 
to (1.2) and (1.3). We then obtain the following 8 (that is, m -- n) 
equations in 15 (that 1s, mn) unknowns. 


(a) zii 4F zi2 d- zia d- zia TF oxi5 — adi 
(b) z2i d- za2 d- x23 MF x24 -bomo5 -— as 
(c) z31-F x32 d- x33 dF r:4 d-oz:5 — a3 
(d) zu T ze Tzu —- bi 
(e) zi? Torn -cF xa - bs 
(n zis -cF zos -J- ras EST 
(g) zi d ozu zu zb. 
(A) z15 TF zs cx: — bs 


If we sum Eqs. (d) to (h) and subtract Eqs. (5) and (c) from this sum, 
the result is Eq. (a). Hence Eq. (a) is redundant and does not need to 
be included in the svstem. . Generalizing, we note that one equation from 
the system (1.2) or (1.3) can be eliminated, and the transportation problem 
reduees to m -4- n — 1 independent equations in mn variables. For the 
preceding equations we have the following matrix for the reduced system 
(here we have eliminated the first equations of (1.2)]: 


Pi Pi; Pi Pi Pi; Pj Pi P5 P4 Ps Püi Pi Pas P4 Ps 
0 Qro E cr - cabe cde d iL TE - o qr 8) — 39) 0 


(1.5) 


EO CONCOCNLC 
e cUeUuMOÀDE 
CUm OO 
(m [ES ESY (Cm) fem m3] 
|-—x Xe de; fem Cumt) 
(eco ee i i e] 
(esr en emt e Ces em 
COUCOUEX COLS CS 
(ey de ens x (e em 
LU CNCECNCONCS 
(esee) es ee m 
[fem e | eu 
eu OoOC x 
[s] elem a c t 
gem) (eei recyuemu 
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and 


P,- b» 


The vector P;; corresponds to the variable z;. Letting 
X -— (211, ORE: mom Gn ts dam) 


be a column vector with mn variables and c — (ei . . . ,6ó . . . , €mo) 
be a row vector of cost coefficients, the reduced transportation problem 
can be represented by 


AX — P, 
X20 (1.6) 
cX a minimum 


From the discussion in Chaps. 3 and 4, we have that the minimum 
feasible solution to (1.6) requires at most m -- n — 1 positive rijs. 


Theorem 2. Construction of a basic feasible solution.! A solution of at 
most m 4- n — 1 positive xij's exists. 


To demonstrate this theorem, we shall use the procedure for obtaining 
a first basic feasible solution proposed by Dantzig (18] and termed the 
"northwest-corner rule" by Charnes and Cooper [10]. We shall apply 
this scheme to the following 3 X 4 tableau: 


Xii. 212. Xi38. X14 | 031 
X21. 222 X28 X24 | Q2 
X31 X32 X38 X34 | Qa 


bi b» bs b, 


! In the paper by Demuth [32d] (translated by Doig [37a]), it is shown that if .N 
is the minimum number of basic feasible solutions to a transportation problem with 
m origins and n destinations (m «X n), then N is bounded as follows: 

qm"! « N € n"-Ymn- (Nondegenerate problem) 

n! 


Qvo d "m Di XS NS nn Imsct (Degenerate problem) 


The upper bounds are due to Simonnard and Hadley ([90c]. 


Sec. 1] THE TRANSPORTATION PROBLEM 213 


We first determine a value for the northwest-corner variable z;. We let 
£,; — min (ajb)); and if a1 € bi, then zii — ai, and all zi; — 0 for j — 2, 
9,4. l1fa, - bi, then zj4 — bi, and all za — Oforz — 2,3. For discus- 
Sion purposes, assume that the former is true; then this initial step 
transforms the tableau as shown in Step 1 below. Here the total left 
to be shipped from origin 1 has been reduced to 0, and the total left to be 
shipped to destination 1 is 0 — a4. 
Step 1l. Assume b; » a. 


gay Ep 0r 20-0 0 


X21 X22 X28 24 a2 


b, — [^21 05 bs b, 


We next determine a value for the first variable in row 2. We let 
2»; — min (as,b, — ai). If weassumeas » 0; — aj, wehavezs — b13— a4 
and z;; — O0. "TThisisshown in Step 2. "The total left to be shipped from 
origin 21s now às — 6, 4- a, and the total to be shipped to destination 1 
is 0. 

Step 2. Assume a» » b, — a. 


X117 Q1 0 0 0 [U 
€£91 — bi — Q1. Xa9. Xos/ £34 | Q3 — bi c- ai 
0 X32 X33 X34 [^7] 
0 bo bs bi 


Similarly, for the following steps, we determine a value of à variable 
z;; and reduce to zero either the amount to be shipped from : or the 
amount to be shipped to j, or both. 

Step 3. Assume as — b d- a1 » bs. 


X11 — O31 0 0 0 0 
X91 — b1—a0a1 zs — b» X23 X24 d$ — bi "p — b» 
0 0 X33 X34 Ur 
0 0 bs bi 


Step 4. Assume a» — b 4- a1 — bs « bs. 


2117 O1 0 0 0 0 
£31 — bi — d X39 — ba  Xs5 — a» — bi -d- a1 — bo 0 0 
0 0 T33 X34 | Q3 


0 0 b; — as --b1 — a1 4-09 bi 
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From Step 4, we see that z353 — 63 — as -- b1 — a1 d- bs and z44 — ba. It 
should be noted that the values for the z;; were all obtained by adding 
and subtracting various combinations of the a; and b;. Hence, if the 
a; and b; were originally nonnegative integers, then the feasible solution 
obtained by the above procedure would also consist of nonnegative inte- 
gers. Since the northwest-corner rule of determining each z;; eliminates 
either a row or a column from further consideration, while the last alloca- 
tion eliminates both a row and a column, this feasible solution can have 
at most m 4- n — 1 positive zs. Depending on the values of the a; and 
b; and on the assumptions made in obtaining the feasible solution to the 
above example, we have as the six variables with possible positive values 


que 3931 —5054—a 
X99 — D» X53 — Qd» — bi d- a1 — b 
X35 — bg — 83 d b3 — a1 4-0» X34 — b, 


We next exhibit two numerical examples and their northwest-corner-rule 
basic feasible solutions. 


Example 1 
2:0 9. 0: 20712 
1 xpo ux 
0 0 / MEE nd 


Example 2 
102008000). es) 
0r ote dy b et 
0 01 (e 9v 8 nf 
IE OS cae A 


In Example 2, only m 4- n — 2 — 6 of the zj/s are positive, and we 
have determined a degenerate basie feasible solution. This will 
happen whenever the amount left to be shipped from origin i is 
exaetly equal to the amount to be shipped to destination 7, except 
when both? — mandj — n. In Example2 thisis true for zs; — min 
(ds — 22; 53) -— (9,9) M0 

Associated with each such solution is a set of m -- n — 1 linearly 
independent vectors. For Example 1, the vectors associated with the 
positive z;; are Pj;, Py;, P»? P», P5, P4, Ps. Let the basis formed 
by these vectors be denoted by B. "Then the solution to the system 
BX — P, must yield the solution obtained by the northwest-corner 
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rule. Here X — (zi,221,22»,223,233,234,235) 18 à column vector. We 
can similarly select an associated set of m 4- n — 1 linearly independ- 
ent vectors for the degenerate solution of Example 2. "This will be 
discussed below. The solutions obtained by the northwest-corner 
rule (and similar schemes) are extreme-point solutions, and only such 
solutions need be considered as candidates for the minimum feasible 
solution. 


Since virtually all applications of the transportation problem require 
the shipping of only whole units of the item being considered, it is use- 
ful to establish the following important property of the transportation 
problem: 


Theorem 3. Assuming the a; and b; are nonnegative integers, then every basic 
feasible solution (2.e., extreme-potint solution) has integral values. 
We prove Theorem 3 with the aid of the following statement: 


Lemma 1. Every set of m 4 n — 1 linearly independent vectors of the 
reduced-transportation-problem system of equations can be arranged into 
a triangular matrix. 
To illustrate this point, let us rearrange the basis of Example 1 
into a triangular matrix. The original system given by BX — P, is 


Pu Pao Ps Pa Pa Pa Pa X P, 
(a) VONT LC LEE E DR SET TEUS 4 
(b) Qo c0 o9 ep oq 1 l[Izz 7 
(d) Qut ug n pre 07 Mtros 109 oae ee Duo 
(e) 0 0 0 1 H 0 O | [23a 4 
(f) o mop Jo vw INR eu 2 
(g) DO $0 occ 59" PU Uo 2 


To rearrange this B, we need only interchange the rows of B and cor- 
responding elements of Po, as shown below. Since the rearrange- 
ment of B does not require any column interchanges, the elements of 
vector X are not permuted. The transformed system is 


Pi; Pa P5 P4 Ps P4 Ps X P, 
(c) l 1 0 0 0 0 om 3 
(a) 0 1 1 1 NEQU e NES 4 
(d) 0 0 1 0 0 0 O0 | |o» 2 
(e) 0 0 0 l l 0 0 ||[zz»|—2|4 
(b) 0 0 0 0 T Di l ||zss di 
(f) 0 0 0 0 0 l 0 T 34 2 
(g) 0 iU 0 0 0 0 l Jis 2 
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From the system of equations determined by the above triangular 
system, we see immediately that rss — 2 and z34 — 2. Then by 
substituting these values in Eq. (b), we have z;; — 3. In a similar 
manner we solve in order Eqs. (e), (d), (a), and (c) to obtain zs; — 1, 
f -2,354-1,2ndz 2. Since the matrix associated with a 
basic feasible solution is triangular and consists of elements that are 
either 1 or 0 and since the solution of a set of equations determined 
by this type of triangular matrix requires only additions and sub- 


tractions, the values of the variables will be nonnegative integers. 


Theorem 4. À finite minimum feasible solution always exists. 


Proof. By Theorem 1 the problem is feasible. Since the coefficients of 
Eqs. (1.2) and (1.3) are nonnegative and since we assume that all a; 
and b; are nonnegative and finite, no zi; can be made arbitrarily large. 
In fact, the z;; eannot be greater than the corresponding a; or bj. 
(We may note that, for any basic feasible solution, at least one zi; 
equals its corresponding a; or bj.) If the a; and b; are integral, then 
à basic minimum feasible solution has finite integral values. 


'The reduced system of m 4- n — 1 equations in mn variables can, of 
course, be solved by the general simplex procedure. However, for even 
small values of m and m, the resulting system of equations becomes 
unwieldy for manual computation. Such systems could possibly be too 
large to be solved efficiently on computing machinery. "This dilemma is 
resolved by the special adaptation of the simplex algorithm to the trans- 
portation problem. This iterative procedure requires the knowledge of 
exactly m -- n — 1 nonnegative variables that are associated with 
m n —1 linearly independent vectors. We know that the vectors 
that are associated with the variables of a basic feasible solution (i.e., 
an extreme-point solution) are linearly independent. If the basic solu- 
tion 1s nondegenerate, it satisfies the requirement of the procedure. "This 
is not true if the solution is degenerate, as is the solution for Example 2. 
If a degenerate solution has k « m 4- n — 1 positive variables, then we 
have to select m -- n — 1 — k zero variables to be in the solution. The 
m 4 n — 1 -— k veetors associated with these zero variables plus the X 
vectors associated with the positive variables must be linearly inde- 
pendent. "This selection is accomplished by the following **e perturba- 
tion" method for the transportation problem (Dantzig [18]). 

Let us refer to the 3 X 4 tableaus used to obtain a first feasible solu- 
tion by the northwest-corner rule. In Disp l, ifz3 - 5. à», then 
neither zs, nor z»» could be positive. Or in Step 2, if 


X2; — b — as — bi d- ai 
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then neither 5? nor zs; could be positive. Whenever situations of this 
sort arise, the number of positive variables in the basie solution is reduced 
by one. "These degenerate cases arise when, in evaluating some z;;, its 
two possible values are equal. We have then encountered a partial sum 
of the a; and b; which is equal to some a; or b; In Example 2 we have 
à degenerate solution occurring because z; — min (ba, a; — bs 4- a1 — b) 
and bs — a$ — bs -- a1 — 51 — 3. 

To avoid these degenerate situations, we have to make sure that no 
partial sums of the a; and b; can be equal to some a; or b;. "Thisis done by 
"perturbing" (i.e., modifying) the values of the a; and bj; We set up à 
new problem where 


üd;— aide xar Ace P EOTE 
and 
b; 7 b, gus T AD eere «t dp om 
b, — b, -4- me 
for e » 0. 


We select an e small enough that the final solution rounded to the 
same number of significant digits as the original a; and 5; will yield the 
correct solution. "This is possible since the computational procedure 
involves only additions and subtractions. Orden [84] proves that any 
e « ó/2m will do, where 8 1s a 1 in the last significant place in the quan- 
tities a; and bj. In practice, eis taken to be 1 X 1077, where d is deter- 
mined by à/2m. (For machine computation, 1t is easier always to add a 
1l to each a; and to add m to 5, in the last significant place that can be 
carried in the computer.) 

Applying the e procedure to Example 2, we have the solution 


1 e 0 0 0 34e 
001 — re 8 2e 0 4 -4-e 
0 2—2e 5-F-3e| 7 Fe 


1i 3 3 2 5 -- 3e 


The basic solution now contains x», — 2e 7» 0. We see that the e deter- 
mines which of two variables with real values of zero should be brought 
into the solution. Here we had a choice of zs, or z55. The e procedure 
does away with any such choice and enables the computation to procede 
without any degenerate solutions.  Theoretically, the selection of either 
234 Or 23; Would have yielded a basis of m 4- n — 1 vectors. As will be 
seen, the computational procedure requires that we need only note which 
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of these zero variables is selected to be in the basie solution. We can 
then dispense with the e procedure and select either one of the two zero 
variables. It seems best to select the one with the smallest cj. No 
transportation problem has ever been known to cycle. 


2. COMPUTATIONAL PROCEDURE FOR SOLVING THE 
TRANSPORTATION PROBLEM 


In [18], Dantzig shows that, for any basic feasible solution, numbers uu; 
and v; can be found such that, for those z;; in the basie solution, we have 
u;-d- vc; JDantzig also shows that, if u; 4- v; — ci for those variables 
not in the basic solution and if all &; — c;; € 0, then the basicfeasible 
solution is also a minimum solution. If this condition of optimality is 
not satisfied, then we can very readily obtain a new basic feasible solution 
whose corresponding value of the objective function 1s less than (non- 
degeneracy assumed) the preceding value. PDantzig's ingenious compu- 
tational procedure enables us to obtain basic feasible solutions without 
setting up the usual simplex tableau and to test for optimality, i.e., to 
compute the z; — cj, without the explicit representation of the vectors 
not in the basis in terms of the basis vectors. (See Henderson and 
Schlaifer [55] and Charnes and Cooper [10] for nontechnieal discussions 
of the computational procedure.) We shall illustrate this procedure with 
a sample 3 X 4 problem. 

In the problem we have three origins, with availabilities of a4 — 6, 
a» — 8, and a; — 10, and four destinations, with requirements of b, — 4, 
b; — 6,05 — 8, and b, — 6. We note that p Q; — »: b; — 24. "The costs 

L J 


between each origin and destination are given in the following cost matrix: 


Destinations 


Origins 2 


For example, the cost for shipping one unit between origin 3 and destina- 
tion 2 i$ cs» — 2. In general, the ci; can be any positive or negative 
numbers. Using the northwest-corner rule, we obtain the first feasible 
solution: 
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(zi) — 


whére 2j; — 4, 215 — 2, $55 — 4, $5. — 4, $55 — 4, 234 — 6, and all the 
other z;; — O0. "The value of the objective function is 42. Since this is à 
nondegenerate basie feasible solution, the first solution does not require 
our perturbing the problem. As we are doing a simple hand computa- 
tion, it is easler to resolve degenerate situations by not employing the 
technique at all. Instead, when a degenerate solution occurs, we shall 
select, from the two variables that can be in the solution with a value of 
zero, the one having the smaller c;. "The example will illustrate this 
point. We shall always keep track of a solution with exactly m 4- » — 1 
nonnegative variables. If the problem were to be solved on an electronic 
computer, it would be more efficient to start the computation with the 
perturbed problem. 

We are next required to determine, for those variables in the basic 
solution, * numbers vu; and n numbers v; such that 


Wyct- tg £g — 1 
uid 7» — €: — 2 
Us d V; — € —38 
WU» d vs — €235 — 2 (2.1) 
Us -F Vs — 633 — 2 

1 


Uus d v4 


I 


(Eryi n 


Here we have seven variables in six (that is, m 4- n — 1) equations. 
Since (2.1) is an underdetermined set of linear equations (1.e., the number 
of unknowns exceeds the number of equations), this system has an 
infinite number of solutions. We determine a solution by arbitrarily 
letting any one of the variables equalits corresponding ci. "This reduces 
the number of unknowns by one and forces à unique solution of the 
m -- n — lequations in the remaining m 4- n — 1 variables. 

In (2.1) letu; — 1;thenitis an easy matter to solve for the remaining 
u; andv; We see that, with u, — 1l, thenv; — Oandvs — l1; with»; — 1 
then u; — 2; with u; — 2 then v; — 0; with v; — O0 then u; — 2; and with 
us — 2 then v», — —1. "This computation can be readily accomplished 
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by setting up the following table containing the cost coefficients (in bold 
type) of the variables in the basic solution: 


By letting u; ^ 1 and v», ^ 0, we can compute, as was done above, the 
resulting wu; and v; and enter them in the corresponding positions as 
follows: 


Since all the equations of (2.1) are satisfied, we have uw; -- v; — ci for 
those z; in the basic feasible solution. We then compute &; — uw; 4- vj 
for all combinations (7,7) and place these figures in their corresponding 
cells of the znd?rect-cost table. (6j; — ci for all z;;in the solution.) The 
indirect-cost table (6;) for the first solution is as follows:! 


! A concise way of combining the allocation table and the indirect-cost table is to 
form the following tableau (shown for a 3 X 4 problem): 


Sec. 2] THE TRANSPORTATION PROBLEM 221 


For example, &j4 — u1 4- v4 — 0. As will be shown below, the above 
three boxes actually ean be combined into one efficient computational 
step. We next compute the differences &; — cj. If all &; — ci; € 0, 
then the solution that yielded the indirect-cost table is à minimum feasible 
solution. If at least one &; — ci; » 0, we have not found a minimum 
solution. We can, as will be deseribed below, readily obtain a new basic 
feasible solution which contains a variable associated with a &j; — c; 7 0. 
As in the general simplex procedure, we select for entry into the new basis 
the variable corresponding to the max (£j; — ci; » 0). "This scheme will 
yield à new basic solution whose value of the objective funetion will be 
less than the value for the preceding solution (or possibly equal to it, if 
the preceding solution is degenerate). 

For our first solution, we find that max (&; — cj) — 6&1 — ca — 2. 
Hence we select z;; to be introduced into the solution. If there were ties, 
we would select the one with the smaller cj. Referring to the first solu- 
tion matrix (rj), we first introduce z;1 into the solution at an unknown 
nonnegative level 0j. As the row and column sums of the variables must 
equal the corresponding values of the a; and 5;, we must add or subtract 
0; from some of the other z;; in the first solution, as follows: 


Since we put 0, 2 0 in cell (3,1), we must subtract 0; from zi;, zs», and 
23s and add 6, to z;; and rss, in order to keep the row and column sums 
correct. "We see that the size of 6, is restricted by those z;; from which it 
is subtraeted. 6; cannot be larger than the smallest x;; from which it 1s 
subtracted. Here 0; must be less than or equal to 4 and must be greater 
than zero in order to preserve feasibility. Since we wish to eliminate one 
of the variables from the old solution and to introduce x31, we let 0; — 4. 
However, since zi; — 01 — s» — 0i — 33$ — 01 — 4 — 06,, we shall elimi- 
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nate three variables and obtain a degenerate solution with four positive 
variables. To keep a solution with exactly m 4- n — 1 nonnegative vari- 
ables, we retain two of these three variables with values of zero. We 
select z,; and 25s, because they correspond to the smaller cj. "The new 
solution is shown below: 


(5). — 


'The objective function for this solution 1s equal to 
42 — [max (& — €g 2 0)]6; —-42— (2) (4) — 94 


'The corresponding combined (u;v;) table and &;; matrix for this solu- 
tion is given in the following table: 


where the bold numbers correspond to the ci; for the basis vectors. We 
see that max (&; — cj 7 0) — && — c»4 — 1. Introducing 6; » 0 in cell 
(2,4), we have to add and subtract 6» from the zi; as follows: 


TZ | ; 

| 8-6 0; 8 
" IE 10 
4 6 8 6 


We have 06» — 6; zs4 is eliminated, and zs, is introduced with a value of 
0; — 6. The new value of the objective function is 


34 — (£24 3 C24)05 — 34 — (1) (6) — 28 
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'The new solution is 


X94 — 0, 3| — 4, x;,; — O0, and all other z;; — 0) is a degenerate minimum 
feasible solution. The value of the objective function is 28. We note 
that €i — ci; — O0 and that zi; is not in the solution. We can then 
introduce z;; into this last solution and obtain an alternate minimum 
solution. We put 03 2 0 into cell (1,3) and obtain 


Here, all &; — c,;; € 0, and this last solution (zxi1 ^ 0, zi» — 6, zs — 2, 


We then have zi; — 0; — 0 and a new degenerate basic minimum solution 
as follows: 


Of course, the value of the objective function is 28. For this multiple 
solution the value of the objective function did not change because 
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4s — €i; — 0, while the values of the variables did not change because 
03 A 0. 

As a constant can be added or subtracted from each element of a row 
or column in the cost matrix without changing the optimum allocation (see 
Exercise 5), the final set of u; and v; elements can be subtracted from the 
corresponding rows or columns of the original cost table. "This results in 
a cost table whose elements are either zero or positive, that is, 


CIRCUM cati [D zx 0 


When this transformed cost table is used for our transportation problem, 
an optimum solution would, of course, be one that allocated shipments 
only to those routes with zero costs. This is just the situation encoun- 
tered in the final allocation determined by the above variation of the 
simplex method; hence the solution is optimum. 

Some transportation problems might have the situation where the 
total of availabilities, » à;, is less than the total of requirements, » b;. 


Even though we SuSE satisfy all the requirements, we can still gllssate 
the items at the origins to the destinations in a manner that minimizes the 
totalshipping eost. For this case, we assume that we have a ''ficetitious" 
origin that has on hand a total of » b; — » a; 2 0 units. "The costs of 


shipping a unit between this fontibus or (m 2: l)st origin and the destina- 
tions are assumed to be zero. If our problem was originally a 3 X 4 
problem, we would set up the following 4 X 4 problem and solve this 
problem with the fictitious origin like any other transportation problem: 


Destinations 


Origins 


By letting all the c,41; — c4; — 0, we make the minimum value of the 
objective function for the fietitious problem equal to the minimum value 
of the original problem. 


If the problem is given with Y Üoze 2 b; we set up a similar ficti- 
i 3i 
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tious problem. Here we assume a fictitious destination that requires 
2 Qi — » b; 2 0 units. The fictitious shipping costs are again zero. 
i 3 


Pora$o X 4 problem we would then solve the following 3 X 5 problem: 


Destinations 


l 


Origins 2 


3 


In solving any linear-programming problem, we should, in general, 
expect the total number of iterations required to depend on how close the 
value of the objective function for the first feasible solution is to the actual 
minimum. Since the northwest-corner rule does not consider the size of 
the cij, we cannot expect the corresponding value of the objective function 
to be close to the minimum. | À number of alternative methods have been 
suggested that can be adapted to machine computation. We shall 
illustrate three of them for the following example taken from Dantzig 
[18]: 


'The northwest-corner solution yields a value of the objective function of 
Dor 

a. Row minimum. Let the minimum element in the first row be cy. 
(If there is more than one minimum element, select the one with the 
smallest index j. We let xi. — aj1if a1 € by or zy — bif a1 5 b,. In 
the first case, we have shipped all the a, units and go on to the second 
row after changing 6, to b, — ai. We next find the minimum element in 
the second row and repeat the process. In the second case, we have 
allocated only b, units of the a;,; hence we change a; to a1 — b, and b, to 
zero and find the next smaHest c;; in the first row and repeat the process. 
Using this scheme, we have a first feasible solution as shown below: 
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(2i) — 


"This is a minimum feasible solution with an objective-function value of 13. 

b. Column minimum. We do a calculation similar to that using the 
row minimum, except that we start with the first column and proceed to 
the last column. . Using this scheme, the first feasible solution is 


(t) 


with an objective-funetion value of 17. 

c. Matriz minimum. Here we search the whole matrix for the 
smallest element and allocate accordingly. "We repeat this procedure 
until all the units are shipped. Using this scheme on the preceding 
example, we obtain the minimum solution obtained by the row-minimum 
procedure. Even though these schemes produce good results for the 
example in question, they are not foolproof. Examples have been con- 
structed whose northwest-corner solution is much better than solutions 
obtained by any of the above procedures. However, experience has 
shown that, in general, the extra computational time required to obtain 
& first feasible solution by one of the above methods will be more than 
made up because the total iterations will befewer. This is especially true 
for large problems. 

For high-speed digital computers, where searching for elements can be 
time-consuming, a number of procedures have been employed for deter- 
mining which variable should enter the basis, i.e., a rule that does not 
select the z;; associated with the max (&; — ci; » 0) for all (2,7). Dennis 
[32e] describes rules for searching the cost data which appear to be very 
effective. In particular, a complete row of the indirect-cost matrix is 
examined and the z;; in this row with the greatest &;; — ci; 7 0 is chosen. 
For the next iteration, the following row is examined in the same way, 
and the procedure continues in this cyclic fashion. 
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3. VARIATIONS OF THE TRANSPORTATION PROBLEM 


a. The personnel-assignment problem.! We shall illustrate the basic 
problem by the following example: A business concern needs to fill three 
jobs, which demand different abilities and training. "Three applicants, 
who can be hired for identieal salaries, are available. However, because 
of their different abilities, training, and experience, the value of each 
applicant to the company depends upon the job in which he is placed. 
The estimate of the value of each applicant to the company each year if 
he were to be assigned to any one of the three jobs is given below. 


Applicants 2 


It is desired to assign the applicants to the Jobs in such à way that the 
total value to the company 1s à maximum. In this problem, there are 
3! — 6 possible assignments. Here we wish to determine the value of the 
assignment xi; of the ?th applicant to the jth job. We can then restrict 
z;; to be either O or 1l. An assignment value of 0 means that the zth 
man does not get the jth Job, and a value of 1 means that the th man does 
get the jth job. Since each man can be assigned only one job and since, 
conversely, each job requires only one man, the total assignment value 
of the ?th man, » z;, must equal l. Similarly, the total assignment 


J 
value associated with each job, o z,, must also equal 1l. If we let the 


contribution of the th man to the common effort 1f he 1s assigned to job 
j be cij then the above example has the following linear-programming 
interpretation: 


3 
xcu 1 
-1 


S EN jo 1.2.9 


12 


à 


and 


CL 
» » e Lg to be a maximum 
1—17-1 


1 'The material in this section is from Dwyer [41]. 
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The form of this problem is that of the transportation problem if we 
minimize the negative of the objective function, and hence it can be solved 
by means of the same computational procedure. 

Frequently there are many identical jobs which demand the same 
basie qualifieations. Such jobs can be combined into a job category. 
We shall assume there are n such categories and let b; denote the number of 
such jobs grouped in the jth job category. If different individuals have 
identical or approximately the same ci; values, then these individuals can 
be grouped into personnel eategories. We shall assume there are m such 
categories and let a; denote the number of men in the :th personnel 
category. Thelinear-programming formulation of this general personnel- 
assignment problem is then 


ECL QUE la gas CSS IT) 
j- 
m 
EN Ti bj Jioc 1, 2, m 
ic1 
Xi 2 0 
and 
m Lo 
Y cts ^ tobea maximum 
i21j-21 


Here, z;; is either zero or a positive integer indicating the number of per- 
sons in personnel category ? assigned to job category j. We imply that 
the total number of jobs available is equal to the total number of men to 
be assigned, that is, that 2 avem »» b; 1f »3 a; « » b;, then a fictitious 


* E 2 : 
personnel category containing »3 b; — » a; men is added to the problem. 
Jj 1 
When Y bj« Y ai, then a fictitious job category containing y die bi 
- ; , 
1 j 


jobs is used. 

Both Dwyer [41] and Votaw and Orden [977] diseuss methods for obtain- 
ing "near-optimal" solutions to the personnel-assignment problem and, of 
course, to the transportation problem. The special problem of assigning 
^ individuals to n jobs, i.e., all a; — 1 and all b; ^ 1, has led to the devel- 
opment of a number of computationally efficient algorithms (Balinski 
and Gomory [36], Kuhn [67a], and Ford and Fulkerson [44a]). Another 
problem which takes on this formulation is the machine-assignment prob- 
lem in which n tasks are to be assigned n different machines with the cost 
or time of setting up the jth machine for the th job being c. The objec- 
tive is to assign the jobs so as to minimize the total set-up cost or time. 
For additional techniques for handling the assignment problem, the 
reader is referred to von Neumann (78] and Egerváry [42]. 
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b. The contract-award problem.! | AMthough the contract-award prob- 
lem is peculiar to government operations, its formulation and solution as à 
linear-programming problem represent an outstanding application of 
linear-programming techniques. Whenever a government agency wishes 
to procure items from civilian sources, producers of the items must be 
invited to participate in the bidding for contraets. Theindividual manu- 
facturer submits bids in which he states: 


1. The price per unit of article or articles 

2. The maximum and minimum quantity of each item that ean be pro- 
duced at the stated price 

3. Any other conditions he wishes to impose 


The bid reflects the manufacturer's desire for profit, his guess about the 
other fellow's bid, and his own peculiar limitations. "The agency must 
award contracts in such a way that the total dollar cost to the government 
is at à minimum. 

In evaluating the bids, the procurement office must add shipping and 
other related costs to each of the bidders' quoted prices. Similarly, 
any savings that could be effected by agreeing to certain conditions are 
subtraeted; e.g., a discount may be allowed for payments made within a 
certain time. Once the contracts have been awarded, the procurement 
office must be ready to demonstrate that the total cost to the government 
was the least possible. "The application of linear-programming tech- 
niques to this problem guarantees that the solution 1s minimum. 

Let us first consider the s?mple contract-award problem. Here there 
are m separate bidders willing to furnish the needs of x supply depots. 
The :th bidder wishes to produce an amount not exceeding a;, for 4 — 1, 
2, ...,»,and the jth depot requires the amount bj, for? — 1,2, ..., 
n. lt costs c;; dollars to purchase and deliver a unit from the ?th bidder 
to the jth depot. If z;; denotes the quantity purchased from the ith 
manufacturer for shipment to the jth depot, then our problem is to find à 
set of nonnegative zi; such that 


DEEST Lupe ,m 
j-21 
». Ti bj dei n 2, n 
1-1 


1 For à more detailed discussion, see Stanley, Honig, and Gainen [91]. 
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This is a problem of the same type as the transportation problem. 
Note that m aj zé » bj In general, we would expect more to be offered 


1 J 
for sale than is required in the total award; that is, 2» d; 2 b; If we 


i J 
establish a fictitious depot, the (» 4- l)st depot, which absorbs the 
amount » Q; — X b; — b,,1, assigning zero costs ci, 441 for all 7, we have 


then formulated Dum simple contract-award problem as a general trans- 
portation problem. The problem is called simple because it does not con- 
sider the many conditions that can be stipulated by the manufacturers. 
For example, bidder one may wish to make a, units but is willing to accept 
an award for any part of a,. Bidder two bids for a» units and will not 
accept a contract forless than a». Bidder three bids for a; units but will 
accept as little as a; « as units. Bidder four is bidding for a total of a, 
units but has different costs on the sublots that total to his final award. 
By using various computational techniques, these and other such con- 
ditions ean be handled while the problem still retains the basie form of 
the transportation problem. Some of these devices are discussed in 
Stanley, Honig, and Gainen [91]. A typical contract-award problem 
and its solution is given in Directorate of Management Analysis [36]. 


EXERCISES 


1. For the transportation problem of Exercise 1, Chap. 1, obtain a first feasible 
solution by means of the northwest-corner rule and the column-minimum and matrix- 
minimum procedures. 

2. Using the northwest-corner solution as the first feasible solution, determine a 
minimum feasible solution to Exercise 1, Chap. 1. Are there multiple minimum 
feasible solutions? 

3. Write the dual problem to the transportation problem of Exercise 2 and show 
that the u; and v; associated with any minimum feasible solution are also a solution 
to the dual. 

4. Solve the following problem (Joseph [63]). Consider four bases of operation 
B; and three targets 7;. Because of differences in aircraft, range to target, and 
flying altitude, the tons of bombs per aircraft from any base that can be delivered 
to any target differ according to the following table: 


Target (Tj) 


Base of Operations (B) 
3 


—- (eii) 


4 
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where ci; — tons of bombs per aircraft. "The daily sortie capability of each of the 
four bases is 150 sorties per day. The daily requirement in sorties over each indi- 
vidualtargetis 200. Find the allocation of sorties from each base to each target which 
maximizes the total tonnage over all three targets. Determine all multiple optimum 
solutions. 

b. Show that the optimum set of z;; to any transportation problem 
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is equal to the optimum set of z;; of the corresponding transportation problem for 
which a constant k has been added to or subtracted from all c;; elements belonging to 
either the same row or the same column of the cost-coefficient matrix. 

6. An assignment problem that cycles (due to B. J. Gassner and L. R. Johnson): 
Given a 4 X 4 assignment problem (i.e., all a; 2 b; — 1) with the cost matrix 


show that the northwest-corner solution of 


(zx;j) —9 


will repeat if the following sequence of new variables is introduced into the basis: 
X13, X42, 332, 241, 243, X21, X31, X24, X23, X134, 334, D21 


In the first iteration let z;3 eliminate z;» and do not allow the variable coming into the 
basis in iteration k to replace the variable that came into the basis in iteration k — 1. 
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For this set of iterations, wu; J- v; — cij — ^ — 2 for all basis changes. For the 
northwest-corner solution, the initial u;s are given by w — (1,3,5,9) and vj's by 
v — (2,4,6,8). Letting u' and v' be the set of u;'s and vj's for the final iteration, note 
that u' — (ui 4- 6^, us 4- 6^, us -F 6A, u4 -- 6A) and v/ 2 (v1 — 6^,v» — 6^4,vs — 6A, 
Qc 6A). 

7. Solve Exercise 6 for the maximum of the objective function by the degeneracy 
procedure of the transportation algorithm. 

8. The transshipment problem is a transportation problem in which each origin 
and destination can act as an intermediate point through which goods can be tempo- 
rarily received and then transshipped to other points or to the final destination. 
Show how this problem ean be formulated and solved as a transportation problem 
(Orden [85a]). HiNm: Set up an enlarged tableau with m 4- » origins and m -4- n 
destinations, assuming costs c;; between all combinations of the origins and destina- 
tions are known. Make sure enough units are available to allow for transshipment. 

9. Prove Lemma 1. 

10. Justify the computational procedure of Sec. 2 by using the pricing-vector 
concept of the revised simplex algorithm. Show that for a triangular basis we can 
compute the components of the pricing vector, and thus the z; — c; elements, as 
deseribed in Sec. 2. 


CHAPTER 11 / GENERAL 
LINEAR-PROGRAMMING APPLICATIONS 


In the first part of this chapter we shall discuss and formulate examples 
of some of the more important (and classical) applieations of linear- 
programming techniques. For purposes of exposition we have grouped 
linear-programming applications into four general eategories: production- 
scheduling and inventory-control problems, interindustry problems, diet 
problems, and network problems.! In each section we shall give a verbal 
statement of the problem and then formulate the problem as a linear- 
programming problem. For some problems this is followed by a solution 
of a numerical example. Although these applications will give the reader 
some idea of the adaptability of linear-programming methods, it is not 
our purpose here to describe the complete range of linear-programming 
applications. Instead, these applications were selected mainly for their 
ability to illustrate the basie techniques of formulating linear-program- 
ming models. It is hoped that these discussions and the exercises will 
enable the reader to investigate and develop new applications. 

In order to demonstrate the versatihty of the linear-programming 
model, we have already compiled (see Sec. 3 of Chap. 1) verbal deserip- 
tions of à number of applications. "This survey 1s designed to inform 
investigators in the various fields of industry, management science, and 
operations research of the areas that have been successfully treated by 
linear-programming methods. 

At this point we should like to stress that there are no set procedures 
and techniques one can apply to determine and test the linear-program- 
ming formulation of à given problem. The casting of à new and com- 
plex problem into a linear-programming form appears to be à process of 
evolution. One's first impulse is to define what appear to be the variables 


! 'The second and third categories are sometimes consolidated into the category of 
product mix, i.e., the combination of raw materials and resources to yield finished 
products. "Theimportant category of transportation problems is discussed in Chap. 10. 
See the Bibliography for a more detailed listing of applications. 
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of the problem and to determine the interrelationships between these 
variables, the resulting constraints, and appropriate objective function. 
Problems based on either actual or test data are then solved, using the 
initial model, and the solutions compared with expected results. These 
studies will in all probability suggest that new variables must be defined 
and introduced into the system and that previously defined variables can 
be eliminated; changes in the objective function and constraints, à 
reevaluation of the test data, ete., may also result. Additional problems 
are then solved, and this process for evolving a correct model continues 
until the investigator is satisfied that his resulting model approximates 
the real situation to an acceptable degree (or, possibly, is satisfied that 
his problem cannot be handled by linear-programming techniques). If, 
in developing the constraints of the problem, it is determined that some of 
the relationships are not linear, we are faced with three alternatives: 


1. Approximate the troublesome expressions by appropriate linear 
functions 

2. Redefine the problem to fit the linear-programming format 

3. Use other techniques to solve the problem 


In selecting the first two alternatives, the investigator should avoid a 
drastic transformation of his problem, for this would tend to dilute any 
valid interpretation of his solution. For the third alternative, we might 
possibly use some of the techniques of nonlinear programming (see 
Chap. 13). 

To simplify the formulation of the problems deseribed below, we shall 
discuss all but the diet problem from the ideal point of view; i.e., we shall 
start with a complete and valid statement of the problem being considered. 


1. PRODUCTION-SCHEDULING AND INVENTORY-CONTROL 
PROBLEMS 


Let us consider the plight of the manufacturer of a seasonal item who 
must determine his monthly production schedule of the item for the next 
12 months. "The demand for his product fluctuates, as is illustrated in his 
sales-foreeast chart for the coming year (Fig. 11.1). The manufacturer 
must always meet the monthly requirements given on the chart. He can 
fulfill the individual demands either by producing the desired amount 
during the month or by producing part of the desired amount and making 
up the difference by using the overproduction from previous months. 

In general, any such scheduling problem has many different schedules 
that will satisfy the requirements. For example, the manufacturer could 
produce each month the exact number of units required by the sales 
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forecast. However, since a fluctuating production schedule is costly to 
maintain, because of overtime costs in high-produetion months and because 
of the costs associated with the release of personnel and machinery in low- 
production months, this type of production schedule is not an efficient one. 
On the other hand, the manufacturer faced with fluetuating requirements 


Total units required 
h3 
o 
e 
[e] 


1,000 
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Month FIGURE 11.1 


could overproduce in periods of low requirements, store the surplus, and use 
the excess in periods of high requirements. "The production pattern can 
thus be made quite stable. However, because of the cost of keeping a 
manufactured item in storage, such a solution may be undesirable if it 
yields comparatively large monthly surpluses. 

Problems of this nature illustrate the difficulties that arise whenever 
conflieting objectives are inherent in a problem. Here the desire is to 
determine à production schedule that minimizes the sum of the costs due 
to output fluetuations and to inventories. For these problems, efficient 
scheduling means the determination and acceptance of a middle-ground 
solution lying between the two extreme solutions—the one that mini- 
mizes surpluses and the one that minimizes fluctuations. The optimum 
production schedule will depend on the relative costs assigned to the con- 
flicting objectives. 

Let us next develop a mathematical model of this scheduling problem. 
At the beginning of the first month the manufacturer has in storage a 
certain amount, say so, of the item available from previous production. 
If the item to be produced is of a new design or style, then so — 0. Let 


z, — the number of units produced in month f, that 1s, the production 
r, —^ the number of finished units that must be available in month t, that 


is, the requirement 
s, — the number of finished units that are not required in month f, that is, 


the storage 
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Owing to the nature of the problem, we have z, 7 0, T; 2- O, and s, -— 0 
for all values of t£. For the first month, the production z; and the previ- 
ously stored items sy must be such that their sum is greater than or equal 
to the requirement r,. "This yields the relationship 


t1 So 5 T3 (1.1) 


If the equality in (1.1) holds, then the storage in the first month, s;, must 
equalzero. lf theinequality holds, then s; 7 0. In either case, we have 


(Dau Sg EUST 
Or 
$;-- 85 — 81 * 71 


For the second month, the production x» plus the previously stored items 
s, must be greater than or equal to the second month's requirement r2. 
We then have 


Q5 -]- 8; 2 T» 
Or 
Qa - 84 — Sp 9 T» 


In general, the production z;, the storage s; and the requirement r, are 
related by 


Xt E Sq Spp (1.2) 


The manufacturer desires to minimize the fluctuations in the produc- 
tion schedule and obtain à smooth pattern of production. The difference 
between any two successive months' production, say z, — z,.4, will repre- 
sent the corresponding increase or decrease in production. Since any 
number has an equivalent representation in terms of two nonnegative 
numbers, we set 


bte pL eR CES 


where y, 7 0 represents an increase in production and z, 7 0 represents 
a decrease in production. From (1.2) and (1.3) our basic equations for 
this production model are 


d, Sii — $0 — T. (1.4 

dic mec ums) - 
wherez, 0,5; 0,9, 2 0,2; 2 0, and£ — 1,2, . . ; n. lfattheend 
of the production year we desire the surplus of finished items to be Zero, 
then we set s, — 0. Depending on the conditions of the model, rz, 7 0 
and s, — 0. 


The manufacturer is, of course, interested in maximizing his profit. 
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As we have seen, his profit will depend on the fluctuations in his produc- 
tion schedule and the associated monthly storage of manufactured items. 
From his cost records in previous years, he knows how much it costs to 
increase production by one unit from month & — 1 to month £ and also 
how much it costs to store one unit for 1 month. Let these costs be a 
dollars and 6 dollars, respectively, where a » 0 and b » 0. "The manu- 
faeturer then wants to minimize 


b aam DET 
(—1 t-1 


If we let a/b — ^, where ^ measures the cost of a unit inerease in output 
relative to that of storing a unit for 1 month, we then wish to minimize 


2 8; » Vi (1.5) 


Jf à value of À is known, it can be substituted in (1.5), and the cor- 
responding linear-programming problem can be solved by means of the 
simplex method. It should be noted that a minimum solution cannot, 
for the same month, /, have both a y; » 0 and a z, » 0. Solving the 
problem in this manner will yield a unique optimum production schedule 
or possibly additional production schedules having the same value of the 
objective funetion. In many instances a value of ^ cannot be deter- 
mined, or the manufacturer wishes to study various production schedules 
associated with a range of values for 4. For these situations an explicit 
value of ^ need not be given, and by means of a modification of the simplex 
method, optimum production schedules for ranges of ^ extending from 
A — 0 to A « --« can be computed.! 

If a is very small, then the corresponding optimal solution will be the 
schedule that requires the most fluctuation and the least storage. If 
b is very small, then the associated minimum solution will be the schedule 
that calls for the most storage and least fluctuation. In the first case we 
are minimizing only storage, and in the second case we are concerned with 
minimizing only fluctuations. 

The following example from Directorate of Management Analysis 
[36] will illustrate the preceding discussion: 

The number of monthly graduates required from an aircraft mechan- 
ies! training school (for a 12-month period) is shown in Fig. 11.2. "The 
total production of mechanies over the 12-month period must equal the 
total required; i.e., there cannot be a surplus or shortage of mechanies at 
the end of the program. 

For this problem, the possible schedules range from the one where 


! See Chap. 8 for a discussion of parametric methods. 
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each man is assigned to duty on the day he graduates (minimum surplus, 
Fig. 11.2) to one where a fixed training load is provided (minimum fluctua- 
tion, Fig. 11.3). "The former schedule, by producing each month the 
graduates as needed, requires no replacement pool but creates the heaviest 
problems in operating the training school. "The second schedule elimi- 
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FIGURE 1.2 Minimum surplus solution. (/rom Directorate of 
M anagement Analysis |36].) 
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FIGURE 11.3 Minimum fluctuation solution. (From Directorate 
of Management Analysis |36].) 


nates fluctuations in training output but puts the greatest burden on the 
replacement pool. An intermediate solution (Fig. 11.4) results in a more 
manageable training program, which at the same time places a lighter 
load on the replacement pool. 

The reader will note that for only 12 time periods, that is, forn — 12, 
the linear-programming model involves 24 equations in 48 variables. 
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Problems of this size are difficult to solve by manual computation. It is 
advantageous to reduce the size of the problem whenever possible, not 
only for manual computations, but also for computations being done on 
eleetronie computers. In some instances, a careful analysis of the prob- 
lem will reveal equations and variables that can be eliminated from the 
computation. "The above production problem is such à problem. 
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FIGURE 11.4 An intermediate solution. (From Directorate of 
M anagement Analysis |36].) 


From Eq. (1.2) we have 


qm, Cg ee e en 
and 
qup. $us 


By substituting the above expressions for z, and z,-11n (1.3), we eliminate 
all the x, from the original model and obtain the following set of equations: 


jp epi eee $eg SET fui (1.6) 
and 
feodo es and S17 To — O 


Equations (1.6) and the objective function (1.5) now represent the 
linear-programming model of the production problem. We have elimi- 
nated one equation from each time period and the 12 variables zr, When 
we solve this reduced system, it 1s an easy matter to go back to the original 
equations and obtain the desired production schedule consisting of the 
z;s. However, since the reduced set of equations does not restrict the 
z, 2 0, we have no assurance that the nonnegative values of y;, z;, and s, 
associated with the optimum solution to the reduced problem will yield 
nonnegative values of rz, Before any such elimination is aecepted, the 
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validity of the reduction must be demonstrated. To do this, we first 
assume that an optimum solution to the reduced system is given. We 
then need to show that the optimum values of y;, 2;, and s, cannot yield 
a negative z, and at the same time be optimal. "The proof is left to the 
reader. 

In addition to the above reduction in the dimensions of the problem, 
this formulation of the production-scheduling problem has also been sub- 
jected to further interpretations which enable one to solve it using much 
simpler computational devices than the general simplex method. Bow- 
man [7] gives a variation of the problem that is written as a transportation 
problem and solved by the corresponding algorithm. Dantzig and 
Johnson [28a] have developed a systematic and rapid graphical method for 
determining the optimal solutions. 

For additional reading concerning scheduling problems, the reader is 
referred to Hoffman and Jacobs [60], Antosiewiez and Hoffman [2a], 
Jacobs [62], Magee [73], Directorate of Management Analysis [36], 
Charnes, Cooper, and Farr [11], and Charnes, Cooper, and Mellon [13]. 


2. INTERINDUSTRY PROBLEMS 


The first application of linear-programming techniques in the field of 
economies was to the area of interindustry or input-output analysis 
(Koopmans [65], Leontief [71], and Morgenstern [75]. By way of intro- 
duction to the general formulation of input-output models, let us consider 
an economy that consists of only the three basie industries—railroad, 
steel, and eoal—and a fourth category of all other industries (Glaser [50]). 
We wish to analyze the interrelationships of these industries in terms of 
sales to each other and to other elements of the economy, as will be 
described below, during a base time period, e.g., 1 year. "This analysis 
can be readily accomplished by reference to the Input-Output Table. 
Each element in the table represents a total sales activity of each 
industry that occurred during the base time period. For example, the 
first row of the table describes the sales of the railroad industry (in terms 
of dollars) to each of the other industries. "The first element of row 1 
represents the total sale of the railroad industry to the railroad industry 
(intra-industry sale); the second element describes the total sale of the 
railroad industry to the steel industry; the third element, the total sale to 
the coal industry; the fourth element, the total sale to the other industries; 
and the fifth element, the total sale of the railroad industry to what is 
termed the final-demand consumers. In general, the final demand con- 
sists of those elements of the economy which consume the various com- 
modities produced but do not contribute, or feed back, to the economy à 
product of their own. In the final-demand category we usually include 
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foreign trade, government operations, and households. The total pro- 
duetion required by the final-demand segments of the economy 1s called 
the bill of goods. "The sum of the sales included in these five elements 
represents the total sales of the railroad industry during the base period. 
A similar discussion holds for the other three rows of the table. By 
columns, we see that each element represents the total purchase of an 
industry from the other industries to support its operations during the 
base period. 

Let us define 


0 «€ z; 2 the total output, in dollars, of industry ? during the base 
period (only active industries are considered) 
0 X z; — total sales, in dollars, of industry 7 to industry 7 during the 


base period 
0 € y; ^ amount of final demand, in dollars, for industry : 


where v. m 152, Vom. 

With these definitions we ean construct the following set of four linear 
equations, which define the interrelationships of our reduced economy, as 
deseribed in the table: 


apri ecd spo sp ce Ae es m 


Voc oq oou U2g coqui 
(2.1) 
a ccEd age c aom qua Maa cS 


Jd4 — 4p — 449 — 4g — X44 — W4 


In the form (2.1), our input-output information explicitly defines the 
relationship between total outputs, total sales, and final demands for a 
given time period. 

In the 1930's Leontief was able to gather the necessary data and write 
sets of equations similar to (2.1) for a 45-industry classification for the 
years 1919 and 1929. He assumed that this linear model of the economy 
for a base time period could be used to analyze the structure of the econ- 
omy in future time periods. Let us denote the x; and z;; that are known 
for the given time period by z; and z;. For the railroad industry we see 
that the ratios 


represent the percentages of input by each of the other industries required 
to produce one unit of output by the railroad industry; for example, 
i5/1, 18 the input of the steel industry necessary to produce one unit of 
output of the railroad industry. 
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Let 0 € a; — — - amount of industry ? which is necessary to pro- 
duce one unit of commodity 7 


The a; are called input-output coefficients. 
For the base time period we have 
$1 —. 4393 —.01»35 — 018433 — (1404 7 0/1 
X2 — QsiXi — G»»f» — Q»3Xa — Go4X4 — do 
X3 — Gaii — Ga»X2 — Ga3X3 — Osa4X4 — da 
X4 — GaUi — Ga»Xo — Ga4sts — GadTa4 — a4 


We can then write (2.2) in the form 


(I— A)X -Y 
where 
23 Ji 
Ao xe Toe equa 
(a) - " 
pU m 


'The matrix (I — A) is known as à Leontief matrix. By assuming that 
this linear structure of the economy, i.e., the input-output coefficients, 
deseribes the activity of the economy not only for the base time period 
but for future time periods as well, we can determine a production vector 
X which satisfies a predicted final-demand vector Y. From this we have 
that the general problem for input-output economies is to find a vector X 
which satisfies the constraints 


X20 


(Ios X my eu 


where Y is a given nonnegative and nonzero final-demand vector and A is 
a given matrix of input-output coefficients. For the problem being con- 
sidered, the one with the nonzero final demand, it can be shown not only 


T 


that a;; 2 0 but also that » 4 «1for;21,2,...,m (Morgenstern 


1-1 
[75]. "This structure is called an open model. It is also shown in 
Morgenstern [75] that, 1f à matrix A satisfies the condition 


DET ES! fon eoe oct rA 
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then (I — A) is nonsingular. We then have that Eqs. (2.3) have the 
solution 


XA cy 
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The initial treatment of the interindustry model as a linear-program- 
ming problem interpreted the equalities of (2.3) as a system of inequalities 


qx xc (2.4) 


Here we are interested in solutions that do not have to satisfy the final- 
demand requirements. An obvious but impractical solution to (2.4) is 
X — O; hence the problem is feasible. An alternate formulation of the 
problem might require the system (2.4) to be a mixture of equalities and 
inequalities. However, for the inequalities (2.4), the corresponding 
system of equalities 1s 


(I— A)X--W-Y (2.5) 


where W is an m-dimensional column vector whose components w; are 
nonnegative slack vectors. "The system (2.5) consists of m equations in 
2m variables. "The objective function can take on various interpreta- 
tions. For example, if c; is the profit per unit of commodity j produced, 
then an appropriate objective function would be to maximize the total 
profit cX, where c — (es, . . . ,c4) 1$ & row vector. "The cost coef- 
fieients of the slack variables are taken to be zero. Another objective 
funetion could call for the solution that maximizes the output of a particu- 
lar industry or combination of industries, e.g., to maximize zi 4- za 4 z;. 

In addition to the constraints (2.3) or (2.4), the interindustry model 
might stipulate that the production (activity) level z; of industry 7 must 
not exceed à known available capacity for production by industry 4i. 
Let us denote these capacity levels by /; and let the capacity-level vector 
L — (hs, . . . ,l4) be à nonnegative column vector. Then X «LL. 
As we are here looking at our economy for a particular time period, e.g., 
a month, we can assume that for some industries a portion of the produc- 
tion in the previous time period was used to stockpile finished units. 
These units will be made available for distribution in the following time 
period and ean be applied to satisfy the new final-demand requirements. 
In our model let so; represent the stock of product 7 available from 
previous production, and let the vector of stocks, So — (so1,8os, . . . ,8o5), 
be a nonnegative eolumn vector. We can then expand the system 
(2.5) into the following linear-programming problem: To maximize 


cx 
subject to 
I—AX-W-Y-S 


XU IL (2.6) 
X EM 
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where U — (uius, . . . ju) 1s a nonnegative column vector and where 
the uw; represent the unused capacity of industry :. If, for some i, 
yi — 8o; € 0, then the final demand for industry £ will be satisfied from 
stocks, and the effective final demand for industry ? during this time 
period is set equal to zero. However, we assume that at least one 
Ui — Soi px 0. 

'The system (2.6) is ealled a statie Leontief model because it considers 
the economy over only a single time period. A more challenging applica- 
tion of the interindustry model is its use in interpreting the behavior of an 
economy over several time periods. We shall next discuss a particular 
linear-programming formulation of a similar but dynamie model, as 
given in Wagner [98]. To do this, we must introduce the following 
changes in our notation: Let » equal the total number of time periods 
considered, and let £ — 1, 2, . . . , n be any particular time period. 
Then for any t we have the nonnegative column vectors 


X; (uum, ...,0») - We production vector 
Y. — (yuye, . . . ,ym) ^ the final-demand vector 
S: — — (SunSm, . . . ,8) — the storage vector due to unused production 


up to and including month £. "These stocks 
are available in month £ 4- 1 
U, — (uuu, . . . jum) — the unused-capacity vector 


We assume that we are given the initial storage vector S, and that the 
basic capacity-level vector L is the same for all time periods. 

'The main difference between our statie and dynamic models 1s that in 
the dynamie model we shall make provision for the expansion of the 
capacity levels of each industry to meet the requirements of future final 
demands. From time period to time period we shall be adding to L a 
nonnegative column vector V;, which represents additional available 
eapacities. "To do this, we need, in addition to the appropriate Leontief 
matrix, (I — A), the knowledge of the corresponding m-dimensional 
square matrix B of capital coefficients. B is à matrix of nonnegative 
numbers in which the th column represents the inputs from each industry 
needed to build an additional unit of capacity for the jth industry. 
Let 


V, — (V9, . . . ,9is) — thenonnegative capacity-expansion vector 


where v;; is the additional eapacity for industry ? in period t£. Then the 
ith row of the product BV;, that 1s, bii -- iovis --- ^ c c P bigis, repre- 
sents the amount of the zth industry's production that is used to build 
additional eapacity in time period ! for all the industries in our economy. 
This production is not available to meet the final-demand requirements. 
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In this model we assume that the additional production capacity is avail- 
able in the next time period (£ 4- 1). We also assume that the matrices 
(I — A) and B are applicable to all the n time periods and that the final- 
demand requirements will be satisfied. We can, for each time period, 
summarize the above conditions as follows: 


(I— A)Xx- S. — Yi -- BV S (2.7) 
1 
XopU eL. »Y, (2.8) 
q-1 
forí-1,2,...,m. Equation (2.7) states that, for any £, the total 


output plus the previous stocks is equal to the final-demand and capacity- 
expansion requirements for output plus the current period's unused 
stocks. Equation (2.8) equates the total used and unused production to 
the sum of the initial production capacity and the previous additional 
increases in production capacity. We can rearrange these equations to 
read 


(I ers A)X, —BM eS m. 

t—1 
X,— » W-U-L (2.9) 

q-1 
For any given set of Y,, L, and S, we can set up the tableau of coef- 
ficents, as shown in Tableau 11.1. Systems of this sort are termed 
block-triangular. Given an appropriate objective function, which eould 
involve production, expansion, and storage costs, the problem could be 
solved by the standard simplex procedure. However, even for just a 
few time periods, the number of equations and variables becomes too 


Tableau 11.1 


(I — A) —B -I 
I I 
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large to be handled efficiently by most of the electronic computers, 
let alone by manual methods. Some work has been done that enables 
one to reduce the amount of computation by taking advantage of the 
bloek-triangular configuration (with its high density of zero elements) 
or to reduce the number of equations by transforming the system. — The 
former is discussed in Dantzig [20] and the latter in Wagner [98]. 


3. DIET PROBLEMS 


In Sec. 2 of Chap. 1 we defined and formulated the basic diet problem in 
terms of à linear-programming problem. We next discuss a particular 
linear-programming model of a diet problem and review its corresponding 
solution to determine whether the formulation of Chap. 1 does apply. 
Such an analysis is typical for linear-programming problems in general, 
and it is recommended when one attempts to describe a complex situation 
by an elementary linear model. "The models of such situations usually 
start as simple ones, and using the solutions to these modest models for 
à base, the investigator 1s able to evolve more realistic models. — To illus- 
trate this process, we shall briefly deseribe various adjustments to a diet- 
problem model that tend to make it a truer representation of the real-life 
situation. 

Historically, the diet problem of Stigler [92] was the first rather long 
and complicated linear-programming problem to be solved by the simplex 
method. Here the problem was to determine what quantities of 77 
foods should be bought in order not only to yield the minimum cost, but 
also to satisfy the minimum requirements of nine nutritive elements, e.g., 
vitamin À, niacin, and thiamine. The resulting purchases were to form 
the complete diet to sustain a person for a year. "The basic linear-pro- 
gramming formulation of this problem was in terms of 9 equations and 
86 variables, including 9 slack variables. "The final solution obtained by 
the simplex method was, of course, a true minimum solution. However, 
since the simplex procedure deals only in terms of basic solutions, only 
nine of the possible foods were represented at a positive level in the mini- 
mum solution. "Thus the diet obtained by linear-programming methods 
ealled for the purchase of varying amounts of wheat flour, corn meal, 
evaporated milk, peanut butter, lard, beef liver, cabbage, potatoes, and 
spinach, and cost $39.67 (for the year 1939). "The solution obtained by 
Stigler using à systematie trial-and-error procedure required only five 
foods: wheat flour, evaporated milk, cabbage, spinach, and dried navy 
beans, and cost $39.93. Such diets, although quite inexpensive, are 
certainly unpalatable over any period of time, and the selection foods 
would do justice to the chief dietician of a slave-labor camp. As Stigler 
points out, "No one recommends these diets (?.e., true minimum-cost 
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diets) to anyone, let alone everyone." He also cites a low-cost diet for 
1939 that was constructed by a dietician and eost $115. The difference 
in eost was attributed to the dietician's concern with the requirements of 
palatability, variety of diet, and prestige value of certain foods. Starting 
with our basic formulation, how can we modify our procedure or model to 
meet these additional dietary requirements? 

Here we have a correct solution to the problem as stated, but this cor- 
rect solution turns out to be unacceptable to execute. In order to over- 
come this defect, we could look for alternate optimum solutions and form 
various convex combinations of these solutions. In this manner we 
would be able to select a diet that included more than nine foods. Or we 
could look at the diets that just preceded the minimumr-cost diet in the 
hope that one of them would be acceptable. Or, finally, we could reevalu- 
ate the linear-programming formulation of the problem to see if it really 
described the problem under investigation. 

To correct the deficiencies inherent in the solution to the original 
problem a reformulation of this linear-programming problem is called for. 
'The new model should enable more than nine foods to be in the minimum 
solution and also take into consideration human taste preferences for 
certain foods. "These elements can be introduced into the model by 
inequalities that force some foods to be into the final solution in at least a 
minimum amount.! We could also determine preference weights and add 
them to the corresponding cost coeffhicients. Another approach to the 
general diet problem that would introduce more variety would be to sub- 
divide the problem into smaller diet problems, each of which would 
involve only a single class of foods. In this process of suboptimizing, we 
might have the problem of selecting the minimum-eost diets for vege- 
tables, or fruits, or meats, and the composite diet would be a solution to 
the general problem. In these attempts to introduce more realism into 
the problem, we suffer a corresponding increase in the cost of the resulting 
diet. If the problem is further constrained, the eost, in general, will 
increase, and it is up to the investigator to determine the relative value of 
dollar cost versus taste and variety.? 


! [t should be noted that such lower-bound inequalities do not increase the size of 
the linear-programming model. For example, if our lower bounds are aà;, then the 
set of inequalities is of the form zr; 7 aj. Introducing nonnegative slack variables, 
wehavez; — y; —ajorz; — y; - aj. Inouroriginal set of constraints we need only 
substitute y; 4- a; for the corresponding z;, and hence the number of constraints and 
variables remains the same as in the original formulation. 

? The reader is referred to work by Wolfe [102] where he solves a modified Stig- 
ler diet problem with a quadratie preference (objective) function. This treatment 
introduces more realism and variety into the optimum diet. Balintfy [3cc] develops 


à linear-programming approach to total menu planning which produces quite accept- 
able diets. 
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There are, however, many situations where the linear-programming 
formulation of the basic diet problem is applicable. "These problems are 
concerned with minimum-eost feed mixtures for farm animals or with the 
mixing of various elements, e.g., chemicals or fertilizers, to meet minimum 
requirements at least cost. "The application of linear-programming 
techniques to these problems 1s fairly straightforward.! To illustrate a 
typieal problem and also to make available to the reader a rather long 
example of a typical simplex computation, we next present the computa- 
tion of a minimum-cost dairy-feed diet as proposed by Waugh [100]. 
The formulation and computation of this problem are due to Goldstein 
[51a] The computations are given in Tableau 11.2, pages 250 to 253. 

This problem involved the selection of 10 feeds to meet the minimum 
requirements for digestible nutrients, digestible protein, caleium, and 
phosphorus. The feeds are listed in Tableau 11.2. "The linear-program- 
ming model consisted of 4 equations in 14 variables, and the computation 
started with a full artificial basis. The costs and variables for the feeds 
are in terms of units of 100 lb. The excess elements are in units of 1 lb 
and have a zero cost. "The minimum solution calls for the purchase of 
18.771 lb of milo maize; 0.06142 lb of exeess phosphorus; 17.020 lb of 
gluten feed; and 58.528 lb of flour middlings, for a total eost of $2.2798. 
In the final section of Tableau 11.2 we have presented onlv the 2; — cj 
elements and the P, column and have left the rest of the elements to be 
computed by the reader. It is evident that these elements are not 
required for the simplex tableau corresponding to the minimum solution 
unless one wants to develop all the multiple minimum solutions. Here 
there are no such solutions. 

Goldstein, in his computations, illustrates three different criteria for 
selecting a vector to go into the basis. "The standard criterion, and the 
one most easily applied— thatis, selecting the max (e; — cj) » O0 element— 
was used in going from the first to the second basis, the fourth to the fifth, 
and sixth to the seventh. "The criterion used in going from the second to 
the third basis was that the vector selected should reduce the value of the 
objective function as much as possible. To simplify the application of 
this rule, the choice was limited to either x; or zio, that 1s, to the two vari- 
ables with the largest 2; — c; elements. In going from the third to the 
fourth basis, the vector was selected so as not only to reduce the value of 
the objective function but also to eliminate an artificial vector. This 
criterion enables the solution to '(come closer" to a feasible solution. 
Finally, as there was some interest in the minimum feasible solution that 
involved no excess nutritive element, variable x3 was chosen instead of 
z14 in going from the fifth to the sixth solution. 


1 For additional diseussions concerning the formulation of realistic feed-blending 
problems, see Brigham [75] and Jewell [62a]. 
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4. NETWORK FLOW PROBLEMS! 


Let us first consider the mathematical formulation of the following prob- 
lem. We are given a transportation network (pipeline system, railroad 
system, communication links) through which we wish to send a homo- 
geneous commodity (oil, freight cars, message units) from a particular 
point of the network called the source node to a designated destination 
called the sink node. In addition to the source and sink nodes, the net- 
work consists of a set of zntermediate nodes which are connected to each 
other or to the source and sink nodes by arcs or links of the network. 
These intermediate nodes can be interpreted as switching or transshipment 
points. We shall label the source node 0 and the sink node m and refer 
to the intermediate nodes by numbers orasnodez,j,k, . . . . Weshall 
designate the arc connecting nodes 7 and 7 by the ordered pair (7,7) and 
assume that the flow of the commodity is directed from 2 to 7j; that is, an 
are is a one-way street. If the flow can also go from 7 to ?, the network 
would include both directed ares (7,7) and (7,7). Each arc can accommo- 
date à nonnegative flow, and we shall assume that each arc has a finite 
upper bound on its capacity which we shall designate fj. If we let z;; be 
the unknown flow from node 2 to node j, then we have 0 € z;; € f. 

The flow of the commodity which originates at the source O0 is sent 
along the ares to the intermediate nodes and then transshipped along 
additional ares to other intermediate nodes or to the sink until all the 
commodity which began the trip at node O0 finally arrives at node m. 
That is, we impose upon the network the condition of conservation of flow 
at the intermediate nodes; what is shipped into a node is shipped out. 
The problem is to determine the maximum amount of flow f which can 
be sent from the source to the sink. We also, of course, wish to determine 
which ares are used and to what extent of their capacity. A typical 
network is shown in Fig. 11.5. "The number on each are represents the 
capacity f; on arc (7,7). 

We shall use the convention that if an arc (7,7) does not exist, we let 
fü — 0; that is, all ares are possible, but flow is restricted to that set for 
which the corresponding f; » 0. Since we require the total flow into a 


! This section is based on lectures by Prof. Robert Oliver, University of California, 
Berkeley, Calif. and Ford and Fulkerson [43a] 


FIGURE 11.5 
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node to equal the total flow out of it, the mathematical model of the 
mazimal-flow network problem is given by the following: Maximize 


f 


subject to 

2) (xoj — x) — f (4.1) 
j 

E (xi Cm Z4) SU Le 0,m (4.2) 
5 

2 m r2 AN E (4.3) 
j 

0 € nj €f; (4.4) 


Equation (4.1) represents the conservation of flow at the source; 7.e., the 
total amount shipped from node 0 minus the total amount shipped back 
to node 0 must equal the total flow in the network; Eqs. (4.2) represent 
the conservation at the intermediate nodes, while Eq. (4.3) represents 
the conservation equation at the sink. "We note that the z;; are non- 
negative variables with finite upper bounds and that z;; — 0, —- 0isa 
feasible solution. Hence, the maximal-flow problem is always feasible; 
i.e., the problem is a feasible linear-programming problem. 

Let us write in explicit form the matrix of the network of Fig. 11.5. 
If we let the ares correspond to the columns of the matrix and the nodes 
correspond to the rows, we obtain the following tableau (ignoring the 
capacity constraints): 


(4.5) 


'The matrix of the equations 1s called the node-arc ?ncidence matriz. For 
such matrices we note that one of the equations 1s redundant. We can, 
for example, drop the last equation which is given by the negative sum of 
the remaining equations. If we were to solve this problem by the simplex 
method we would couple m equations from (4.5) with the corresponding 
upper-bound constraints and apply the simplex algorithm for bounded 
variables. "The objective function would be to maximize f. However, 
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because of its simple structure, more efficient procedures, to be discussed 
below, are available. "The reader will note that any problem which takes 
on the form of a node-arc incidence matrix can be interpreted as a network 
problem; i.e., each column has one -F1 and no more than one —1. 
Before describing the procedure for solving the maximal-flow problem, 
we shall introduce some appropriate concepts from the theory of networks 
(Ford and Fulkerson [43a], Busacker and Saaty [85], and Simonnard [900]). 

Formally, a network is à collection of elements ?, 7, k, . . . ealled 
nodes and a set of ordered pairs (7,7) called ares. We shall discuss only 
directed networks; i.e., for an arc (7,7) the direction of flow is restricted to be 
from toj. "The network can also have the arc (7,7). A sequence of arcs 
G,3), Gk), . . . , (p,q) is called à chain which connects node ? to node q. 
If? — q, then the chain is termed a cycle orloop. If weignore the direction 
of an arc, then we call an und?rected chain or path from $ to q any sequence 
of ares which form a connected set of arcs from 1 to q. 

Let all the nodes of à given network be divided into two disjoint 
subsets S and S; i.e., a node will be either in S or S, but notin both. In 
addition, let (S,S) denote the set of arcs that lead from those e S to 
those j e S, where the symbol e means "contained in." For example, in 
Fig. 11.5,1£ 8 — (0,],2) and S, — (8m), then (8,9) — (0,3), (1,3), (8,8); 
(1,m), (2m). A cut is the set of ares (S,S) with 0€ S and m e S.T. For 
Fig. 11.5, S ^ (0,1,2), and S — (3,m), the cut is shown by dotted lines 
in Fig. 11.6. A network has a finite number of euts. Another cut for the 


Sr 92) 


FIGURE 11.6 


given network is (0,1), (0,2), (0,3), as shown in Fig. 11.7; here S — (0) 
and S — (1,2,3,m). "We define the capacity of a cut f(S,S) to be the sum 
of the individual capacities of all the ares in the cut; that is (5 S) e »s "n 
ies 
LI LI E je$ 
The eut in Fig. 11.6 has a capacity of 8, while the cut in Fig. 11.7 has a 
T An alternative definition of a cut is that it is a set of directed arcs such that 
every chain from the source node to the sink node contains at least one are of the cut. 


Hence, if all the ares of a cut were deleted from the network, there would be no chain 
from 0 to m. 
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capacity of 6. Since if we remove a cut from a network there would be 
no chains from source to sink, i.e., the maximal flow would be zero, the 
maximal flow cannot exceed the capacity of the minimal eut capacity. 
The cut of Fig. 11.6 limits the flow to no more than 8, while the cut of 
Fig. 11.7 limits the flow to no more than 6. "The following theorem, the 
maz-flow min-cut theorem, establishes the existence of the maximum flow 
whose capacity 1s exactly equal to the minimal eut eapacity. For our 
example, this 1s a flow of 6. 


ESAUCN m) 


FIGURE 11.7 


Max-flow min-cut theorem. For any network the maximal-flow value from 
node 0 to node m 1s equal to the minimal cut capacity. 


Proof. 'The proofisgivenin two parts. We first show that the maximum 
flow is less than or equal to the minimum cut and then show that à 
flow exists which is equal to the minimal eut capacity. 

For any eut (S,S) we sum Eqs. (4.1) and (4.2) for all eS to 
obtain 


pe 2 (3, (a — 232) 


'This can be rewritten as 


1eS eS ie ies 
jeS jeS jeS jeS 
Or 
y -- » X44 — » Tj ss ( Xuj — js vj) (4.6) 
des 
s. ud 9 jeS 


As both indices ? and j of the term in parentheses range over the 
same set of values, this term will be equal to zero. We then have 
Ue » Tig — »» Vji (4.7) 


eS Tes 
jeS JeS 
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Since each flow z;; 2 0 and each zi; € fij, (4.7) becomes 


Jus 2 Sos du e 09S) 
je ied (4.8) 
jeS jeS 
and hence, the maximal flow is bounded above by the capacity of the 
arbitrary cut (S,S), and thus f must be bounded above by the minimal 
eut capacity; that 1s, f «€ min f(S,S) for all possible cuts. 

We next establish that for some flow and corresponding eut 
(to be defined), the equality of (4.8) holds. Since all are capacities 
are finite, à maximum flow exists, although its value could be zero. 
Let the value of the maximal flow be f' and for this f' and correspond- 
ing are flows z;; we define a eut (S',S") as follows. 

Let S' include the source node 0. For any £e S' if zjj sc duh 
thenj e.S'. In addition, if £e $' and 24 7 0, then; eS. Let 5"be 
all nodes not in S". We first note that the set of nodes S' includes 
the sink node m. If m & S', that is, both nodes 0 and m are in S', 
then we can find a path from 0 to m such that for any arc in the path 
either f;; -- xj » Oorzj; »0. Wecould then increase the flow from 
0 to m by adding à small amount to those ares for which f;; — z;; » 0 
and subtracting the same amount from those ares having rj; » O. 
Hence, we would have a flow from 0 to m which would be greater than 
f', contradicting the assumption that f' is the maximal flow. "There- 
fore m e S', and (S',S/) is a cut. We next note that z; — fj; when 
i e€ S' and j e S, and zj; 2 O forze S' and j e S'. This results from the 
definition of S' and S'. "Then for the cut (S/,S/), Eq. (4.7) becomes 


f—5u5y— zm my Yjfgejf08,8) 

ie 1e ies' ieS 

je8' jeS' je8' jeS' 
But from (4.8) f € f(S,S) for all euts (S,S) and since f — JUS S^ 
then (S',53') must yield the minimal eut eapacity and max f — 


J enin) e 8^ S^ 


At this point it is instruetive to look at the dual problem of the 


maximal-flow problem. Recalling that f is a nonnegative variable and 
letting v; be the dual variables associated with the ;th node constraint 


given by (4.1), (4.2), and (4.3), and w;; be the dual variables associated 


with the are-eapacity constraints (4.4), the dual problem as given by 
the node-arec incidence matrix is as follows: Minimize 


D D f wi 
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subject to 
NUES za 
m Eja all (7,7) 
wi 0 all (7,7) 
The rz; are unrestricted as to sign as they correspond to primal equations. 


If we know a minimal eut (S,S), we can construct the optimal solution 
to the dual problem as follows: 


T; — 0 forie S 
T;—-1 for ieS 
wg -1 for (5,3) e (S,S) 
wi -—0 for (7,3) e (S,S) 


(4.9) 


This is a feasible solution, and the minimum value of the dual objective 
function 1s equal to the maximum value of the primal objective function. 
As pointed out in Ford and Fulkerson [43a], if we drop the redundant 
source equation, 1.e., take co — 0, it can be shown that all extreme-point 
solutions of the dual are of the form given by (4.9) for some S with 0 e S. 
Hence, the max-flow min-eut theorem would result from the duality 
theorem of linear programming. 

We next describe an algorithm termed the labeling method for solving 
maximal-flow problems. The process converges in a finite number of 
iterations if the are capacities f;; are all rational numbers. Its validity 
is based on the max-flow min-cut theorem. The steps of the algorithm 
are given below and are designed to find a path over which a positive 
flow ean be sent from source to sink. "The steps are repeated until no 
such path can be found. 


M aximal-flow algorithm 

sTEP ]l. Find an initial feasible solution for the network. "We may, 
for example, begin with all z;; — O. 

sTEP 2. Start with the source node 0 and give it a label [—,» |]. 
The general label for any nodes : and j is indicated by [7 € ,»j], with v; being 
a positive number representing a change in flow between ? and J, ?-4- 
representing an increase of flow by the amount v; from 7 to 7, and ?— 
representing a decrease of flow by the amount v; from j to ?. The source 
label [—,« ] indicates that an unlimited amount of commodity is available 
at the source for shipment to the sink. 

sTEP 3. Select any unlabeled node i. Initially only node O0 is 
labeled. 

a. For any unlabeled node j for which z;; € fj, assign the label 
[;2-,vj] to node j, where v; — min (v; f; — x4). "This limits the amount 
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sent from 4 to j either the amount already sent to ? or the remaining 
capacity of the are connecting 7 to j, whichever is smaller. 

b. To all nodes j that are unlabeled and such that zj; 7 0, assign the 
label [(/—,v;], where v; — min (v; z;j). This enables us to reroute a flow 
going into away from i. (For example, in Fig. 11.8 if we started with 


[0T FIGURE 11,8 


the initial solution zq - 2, 21» — 1, 2:0 — 1, T1, — 1, 2», — O0, we have, 
starting with the origin label [—,2], that the label for node 2 should be 
[0—,1] since v; — min («, z»; — 1) — 1.) 

As node j receives a label, it is processed through Step 3 until all 
labeled nodes have been looked at with respect to the unlabeled nodes 
connecting them. Either this process will bring us along a path to the 
sink node which then gets a label (Step 4), or we cannot find such à 
path which leads to the sink (Step 5). 

STEP 4. "The sink node has received a label [i 4,v,] with ? z£ m and 
7, 2 O0 by the selection rule forv,. Hence we have found what is termed 
a flow-augmenting path and ean add or subtract v, to the ares of the 
path leading from 0 to m. — The new feasible solution is 


quom $5 Us for (7,7) in path and 7 has label [7 4,7; 
fa 7 X4 — Um for (j,7) in path and j has label [2 — ,v;] 
T; X for (2,7) not in path 


The labels are all erased, and we repeat the process starting with Step 2 
and the new feasible flow z;;. 

STEP 5. "The process terminates in that we cannot find a path from 
source to sink. "The maximal flow f is equal to the sum of all the v, 
generated in the applications of Step 3, assuming initial solutions of all 
T;; — 0. To show that this flow is maximal, we let the set of labeled 
nodes in the final iteration correspond to the set of nodes S' in the proof 
of the max-flow min-eut theorem. 

We illustrate the algorithm by finding the value of the maximal flow 
for the network of Fig. 11.5. 'Welet the initial feasible solution be zig -—0 
forallZ,;. Weindicate the current feasible solution and the arc capacities 
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by the ordered pair of numbers (z;j, f;) attached to each arc of the network 
as shown in Fig. 11.9. "The node labels are indicated by [;--,»]. Along 
the flow-augmenting path (0,1,2,m) of Fig. 11.10 we can send an additional 


two units of flow (v, — 2) which yields a new feasible solution as shown 
in Fig. 11.11. We again find an unsaturated path as shown by the labels 


FIGURE 11.9 


FIGURE 11.10 


FIGURE 11.11 


FIGURE 11.12 


of Fig. 11.11. Here v, — l. The new feasible solution which sends a 
total of three units from source to sink is shown in Fig. 11.12, along with 
a new path over which an additional unit can be sent. "The resulting 
feasible solution is shown in Fig. 11.13, along with a new unsaturated 
path for which v, — 2. "The reader will note that the finding of the 
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labeled path of Fig. 11.13 required an application of the rerouting Step 3b. 
The corresponding feasible solution is shown in Fig. 11.14. 'This solution 
has a flow of six units and is optimum as we cannot label any node except 
the origin. The corresponding cut is given by arcs (0,1), (0,2), (0,3) and 
has a eut capacity of 6. 


FIGURE 11.13 


FIGURE 11.14 


À number of modified maximal-flow network problems ean be formu- 
lated in terms of the basie network model described above. For example, 
if à network has two (or more) sources, it can be recast into a single- 
source network by joining the given sources to a third source by artificial 
arcs which have infinite capacity, and similarly if we have multiple sinks. 
The added source or sink is identified by 0 or m, respectively. Another 
variation is found when in addition to the capacities on the ares, we also 
have node capacities. "These latter capacities could, for example, repre- 
sent the maximum amount of the commodity that can be processed by the 
facilities at the node. We transform this problem into a maximal-flow 
problem by replacing node 7 with capacity k; by two nodes :?/ and 4", 
where node 7' is connected to the network by the same ares going into 1, 
nodes ?' and 2" are connected by a directed are from i' to ?"" with capacity 
k; (that 1s, f»;^ — k;), and node ?" is connected to the network by the arcs 
going out of 1. 

Another important class of network problems is the set of minimal- 
cost network-flow problems. | As we shall see, this class of problems includes, 
among others, the transportation problem and the shortest-route problem. 
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We shall first develop the general formulation and then specialize it to 
particular problems. 

For the minimal-cost flow problem we are given, as in the maximal- 
flow problem, à general network over which units of a homogeneous 
commodity are to be shipped from the source to the sink. — Associated 
with each arc (7,7) 1s a cost ci; of shipping one unit of the commodity from 
node i to node j.! We must ship a given quantity of F units from the 
source to the sink so as to minimize the total cost of shipping the P units 
to the sink. Assuming conservation of flow at the intermediate nodes 
and that the flow z;; along an arc is bounded by both upper- and lower- 
capacity restrictions, 0 € L;; X z;; € f;j, the linear-programming model of 
the minimal-cost network problem is given by the following: Minimize 


Dem 
2 


subject to 
» (zo — 2j) — F (4.10) 
» (m;—2)-0 70m (4.11) 
2, Gv — zw) - —F (4.12) 
0xl;Xzg € fs (4.13) 


A typical arc of a minimal-cost network is identified as shown in Fig. 11.15. 
We note that for a given F we have no assurance that the problem 1s 


ij, fij, 5) 
FIGURE 11.15 


feasible. It is usually assumed that all coefficients are integers. We 
shall not describe any of the methods for solving this minimal-cost network 
problem as their development and explanation 1s rather involved. The 
most general algorithm, termed the *''out-of-kilter" algorithm, 1s due to 
Fulkerson and is described in Ford and Fulkerson [43a] and Simonnard 
[900]. 

If we let the network be a roadmap with the source being the origin 
city, the sink the destination city, and the c;; being distances between 
eities (intermediate nodes), we can convert the above problem to that of 
finding the minimum distance between the origin and the destination by 
letting F 2 1l and all/; 2 O andf;; — 1l. "This problem can, of course, be 


1 We assume a linear cost relationship. 
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solved by the regular techniques of linear programming, but more effi 
cient, specialized algorithms exist, for example, the simple combinatorial 
labeling scheme (Ford and Fulkerson [43a]; see also Dantzig [16c]). "This 
procedure, described below, also yields the shortest routes from the source 
to all other nodes. 


Shortest-route algorithm 

sTEP l. Assign all nodes a label of the form [— ,z;], where the first 
component indieates the preceding node in the shortest route and m; 
indieates the shortest distance from node 0 to node. Node 0 starts with 
a label of -; — 0, and its index is always — ; all other nodes start with 
T, — vo. 

sTEP 2. For any are (7,7) for which m; -- ci; € v;, change the label 
of node j to [Z, v; 4- c] and continue the process until no such arc 
can be found. In the latter situation the process is terminated, and the 
node labels indicate the shortest distance from node 0 to node j. 

Ford and Fulkerson [43a] prove that the algorithm will terminate 
under the assumption that the sum of the costs around any directed cycle 
is nonnegative. The simple example of Fig. 11.16 illustrates the algo- 


[0,5] 


FIGURE 11.16 


rithm. The labels can be readily checked as being those which yield 
the minimum distances from the origin to the corresponding node. The 
minimum distance is 11. 

We next transform a transportation problem into a special network 
termed b?partite network.! For exposition purposes, let us consider a 
partieular 2 X 3 transportation problem with costs c;;, availabilities iy 
and requirements b;, with Ea; — Zb;, as shown in Tableau 11.3 (see Chap. 
10.) Here we use triangles and squares to distinguish between the original 
origins and the destinations. "The minimal-flow network of harc wi. 


! A bipartite network is a network in which the nodes can be divided into two sub- 
sets such that the arcs of the network join the nodes of one subset to the other. In 
Fig. 11.17, we have a bipartite network if we omit nodes 0 and m. 
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represents an equivalent statement of the above transportation problem. 

In addition to the method of Chap. 10, many special and computa- 
tionally efficient algorithms for solving the transportation problem and 
the related assignment problem are available (Ford and Fulkerson [43a], 
Kuhn [67a], and Simonnard [905]). Other special network problems 


Tableau 11.3 


include the following: (1) The capacitated transportation problem, which 
is a standard transportation problem with the added set of upper-bound 
constraints 0 € z;; € f; for all shipments between origin ? and destination 
j (Ford and Fulkerson [43a]). (2) The multicommodity network-flow prob- 
lem in which more than one commodity can flow through the same net- 


(0, (Cha Gn) 


FIGURE 11.17 


work. Each commodity has a separate origin and destination and the 
capacity of an are cannot be exceeded by the sum of the individual flows of 
each commodity through that arc. In addition, each are has specified 
capacity restrictions for each type of commodity. The object is to maxi- 
mize the sum of the values of all the simultaneous flows (Ford and Fulker- 
son [43a] and Hu [615]). (3) A network with gains is a single commodity 
network in which the flow z;; sent from node ? to node j along arc (7,7) 1s 
multiplied by a factor k;; before it reaches node Jj; that is, kir; is the 
amount which arrives at node j. "The multiplicative factor could repre- 
sent the loss of a fluid due to evaporation or the gain of à product due to 
breeding (Jewell [620]). 
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EXERCISES 
1. Solve the following production-scheduling problem, using the reduced set of 
equations with the initial conditions s.; — 0, $& — 0, ro — 0. Theobjective function 


to be minimized 1s 


4 5 
l6 -- Y s 15) v 


t-1 01-1 


and requirements are r; — 5, r; — 12, rs — 6,7, — 6,7; — 8. 
2. Formulate the following problem from Jacobs [62] as a linear-programming 
problem: A caterer knows that, in connection with the meals he has arranged to serve 


[ 


during the next n days, he will need r; 2 0 fresh napkins on the jth day, with j — 1,2, 

. ,n. Laundering normally takes p days; i.e., a soiled napkin sent for laundering 
immediately after use on the jth day is returned in time to be used again on the 
(j -- p)th day. However, the laundry also has a higher-cost service which returns 
the napkins in q € p days (p and q integers). Having no usable napkins on hand or 
in the laundry, the caterer will meet his early needs by purchasing napkins at a cents 
each. Laundering expense is b and c cents per napkin for the normal and high-cost 
service, respectively. How does he arrange matters to meet his needs and minimize 
his outlays for the n days? 

3. Given the following 3 X 3 input-output coefficient matrix: 


0 054 059 
A-[0.2 0 0.5 
Oe 0s22 0 


Our three industries and their unknown production vectors X, unknown bill-of-goods 
vectors Y, unknown final-storage vectors Si, and known initial-storage vectors S, are 
related by 


(AX o-Y pens 


The production vector is constrained by a known capacity vector L, that is, 


Nc 
Here 
20 10 
Ss 0 and Top eno 
[uU 10 


Write out the explicit equations of the system in terms of the three industries and 
determine values of X, Y, and S; that maximize cY, where c — (1,5,15). Here the 
problem is to determine not only a production vector but also a corresponding ''opti- 
mal" bill of goods. 

4. Formulate the following *'trim problem" as a linear-programming problem and 
solve the exercise, both as a standard and an integer-programming problem. The 
description and numerical problem are from Eisemann [41a]. Also see Paull (Bibli- 
ography, Sec. 3f) and Gilmore and Gomory [49a]. 


A problem of primary significance to a variety of industries is the suppression of 
trim losses in cutting rolls of paper, textiles, cellophane, metallic foil, or other 
material, for the execution of business orders. "We define the measurement taken 
between the 2 circular ends of a roll as its width," the measurement perpendicular 


Sec. 4] GENERAL LINEAR-PROGRAMMING APPLICATIONS 267 


thereto when completely unwound as its "length." A number of cutting machines 
are at our disposal, the knives of which ean be set for any combination of widths 
for which the combined total does not exceed the overall roll width. "The knives 
then slice through the completely wound rolls much in the same way as a loaf of 
bread is sliced. Rolls are marketed in several standard widths. Orders specify 
desired widths and may prescribe either the number of rolls, or alternatively, the 
total length to be supplied for each width. In fitting the list of orders to the 
available rolls and machines, it is generally found that trimming losses due to odd 
end pieces are unavoidable; this wasted material represents a total loss, which may 
be somewhat alleviated by selling it as serap. "The problem then consists in 
fitting orders to rolls and machines in such à way as to subdue trimming losses 
to an absolute minimum. 


As a numerical exercise, assume that two machines A and B are available; machine 
A is capable of cutting a standard roll of 100 in., and machine B is capable of cutting 
& standard roll of 80 in. "The following orders of various widths must be cut: 45-in. 
roll, 862 rolls; 36-in. roll, 341 rolls; 31-in. roll, 87 rolls; and 14-in. roll, 216 rolls. We 
assume that as many standard rolls as necessary are available and that, excluding 
the unavoidable trim loss, only the widths (that is, 45, 36, 31, 14 in.) on order are cut. 
(HiNm: Let the variables of the problem be equated to possible settings of the cutting 
blades; e.g., on machine A one setting of the blades would cut one 45-in. roll, one 
31-in. roll, and one 14-in. roll, with a trim loss of 10 in. In order to ensure feasibility, 
the formulation must produce at least the number of each roll required.) 

b. Formulate the following as a linear-programming problem (Magee (73]): 

Consider a company that has one production line upon which it produces a single 
homogeneous commodity. We suppose that the commodity sells for a fixed unit 
price; that the costs for regular-time production, overtime production, and storage 
are known and vary between time periods; that the rate of production per unit time 
is known; and that an accurate sales forecast in the form of demand during each of a 
number of successive time periods is known. It is desired to formulate a production 
schedule that will meet the sales forecast and minimize the combined costs of produc- 
tion and storage. 


i — number of time periods 
r; —» number of units of finished product to be sold during :th time period 
S9 — initial inventory 
m; — maximum number of units that can be produced each time period on regular 


time 
n; — maximum number of units that can be produced each time period on overtime 
à; — cost of storage of one unit of product during time period ? 


c; — cost of production of one unit on regular time during time period : 
d; — cost of production of one unit on overtime during time period 7 

z; — regular-time production during ith time period 

yi — overtime production during ;th time period 


For concreteness, it will be assumed that each time period is 1 month in length and 
that the inventory for each month is taken on the last day of that month. This is 
equivalent to adding the month's production to inventory and withdrawing the 
month's sales from inventory at the end of each month. 

6. The multidimensional transportation problem. | Formulatethe following example 
of a three-dimensional problem (Schell [(90]). Consider a soap manufacturer who has 
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l factories in various parts of the country. Each of the / plants can manufacture m 
different types of soap. The soap is to be distributed from the factories to n different 


areas. Let 


aij — required number of units to be shipped from factory ? to area k 

bj, — required number of units of type j to be shipped to area k 

d;; — required number of units to be shipped from factory ? of type j 

rij — amount of the jth type made in the ith plant shipped to the kth area 
cijk — cost of shipping one unit of the jth type from the :th plant to the kth area 
zik 20 


ll 


Haley [52e] gives a computational procedure for solving this problem. There is no 
guarantee that a solution will be integralif the a;;, b;;, and d;; are integral (Motzkin [76]). 

7. Show how the caterer problem (Exercise 2) can be formulated and solved as a 
transportation problem (Garvin [45a] and Prager [855]). 

8. The machine-loading problem.  Formulate the following problem. We are 
given m machines and » products and know the time a;; required to process one unit 
of product j on machine ?. Let z;; be the number of units of product j produced on 
machine 1, b; the total time available on machine 7, and d; the number of units of 
product j which must be processed. We also know the cost (or time) c;; of processing 
one unit of product j on machine i. (We assume each product needs to be processed 
on only one machine.) We wish to minimize the total cost. "This problem can be 
looked at as a generalized transportation problem, and special algorithms exist for 
solving it (Eisemann and Lourie [415] and Jewell [625]). 

9. Form the dual of the shortest-route problem of Fig. 11.16 and show that the 
final v; of the Ford-Fulkerson algorithm represent an optimal solution to the dual. 

10. Develop the equations for a two-commodity network with three nodes with a 
separate origin and destination for each commodity. 

11. Construct an example to show that a multicommodity network with integer 
parameters can have a noninteger optimal solution. 

12. Find the shortest-distance route from the origin to the destination in the 
following network: 


FIGURE E12 


13. The transshipment problem is a generalized transportation problem in which 
each origin can ship to all other origins as well as to all destinations and a destination 
can in turn ship goods to all other destinations and all origins (see Exercise 8, Chap. 10). 
rem how to convert it to a minimum-cost network problem (Orden [85a] and Garvin 
45a]). 


14. Show how the caterer problem can be interpreted as a network problem 
(Ford and Fulkerson [43a]). 
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15. Writeout the dual to the general minimum-cost network problem and develop 
the corresponding complementary slackness relationships. 
16. a. Writeout the explicit equations of the maximal-flow network shown below. 
b. Solve the problem using the maximal-flow algorithm. 


FIGURE E16 


17. Develop the mathematical formulation of the network-with-gains problem 
(Jewell [(625]). 


CHAPTER 12 / LINEAR PROGRAMMING 
AND THE THEORY OF GAMES 


Like linear programming, the theory of games can be considered as a 
modern development in the field of mathematics. To the casual observer 
this would appear to be the only element which these areas have in com- 
mon. For whereas, in the general linear-programming problem, we are 
concerned with the efficient use or allocation of limited resources to meet 
desired objectives, in the theory of games we are interested in develop- 
ing a pattern or strategy of play for a given game which will enable us to 
win as much as possible. It is not until we develop the corresponding 
mathematical statement of a game-theory problem that the remarkable 
correspondence between these two problemsisrevealed.  Thisrelationship 
was first established by von Neumann and Dantzig. In this chapter we 
shall demonstrate the equivalence between the linear-programming model 
and the mathematical model of the important class of zero-sum two- 
person games. 


1. INTRODUCTION TO THE THEORY OF GAMES 


The main concern of the theory of games is the study of the following 
problem stated by von Neumann:! "If n players Pi, Pj, . . . , P, play 
a given game T', how must the ?th player, P;, play to achieve the most 
favorable result?" 

In interpreting this basie problem, we say that the term game refers 
to a set of rules and conventions for playing and a play refers to a particular 
possible realization of the rules, i.e., an individual eontest. We shall 
assume that, at the end of each play of T, each of the players P; will 
receive an amount of money v;, called the pay-off to player P;. We then 
assume that, in order to receive the most favorable result for himself, 
the only object of a player will be to maximize the amount of money he 


! See Kuhn [67]. 
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receives. In most parlor games, e.g., poker, the total amount of money 
lost by the losing players is equal to the total amount of money won by 
the winning players. Here we have 


Qi pote p tre ug um 


for all plays. Note that v; can be positive, negative, or zero, where 
v; 2 O represents a gain, v; « 0 represents a loss, and v; — 0 represents 
no pay-off. Games whose algebraic sum of the pay-offs is equal to zero 
are called zero-sum games. In zero-sum games, wealth is neither created 
nor destroyed by the players. An example of a non-zero-sum game would 
be poker in which a certain percentage of the pot is removed for the 
"house" before the final pay-off. 

Games are also classified by the number of players and possible moves. 
Chess is a two-person game with a finite number of possible moves, and 
poker is a many-person game, also with a finite number of possible moves.! 
A duel in which the duelists may fire at any instant in a given time interval 
is à two-person game with an infinite number of possible moves. Games 
are further characterized by being cooperative or noneooperative. [n 
the former the players have the ability to form coalitions and work as 
teams, while in the latter each player 1s concerned only with his own 
result. 'lTwo-person games are, of course, noncooperative. We shall 
concern ourselves only with finite, zero-sum, two-person games. 

As an example of a finite, zero-sum, two-person game, let us consider 
the familiar game of matching pennies. To accomplish one move, which 
for this game is equivalent to a play, the first player P, selects either 
heads or tails, and the second player P;, not knowing P;'s choice, also 
selects either heads or tails. After the choices have been made, P» pays 
P, 1 unit if they mateh or —1 unit if they do not. 'The pay-off of —1 
represents the giving of 1 unit by P; to P». "The pay-off values of 1 and 
— ] are written in terms of a gain being a positive value to P; and a loss a 
negative value to P,. When this is the case, we say that P; is the 
mazimizing player and P; the minimizing player. We shall always con- 
sider P, as being the maximizing player and write the pay-off values 
accordingly. 

The above statement of the problem can be summarized by the 
following diagram (Kuhn [67]): 


P Selections 
Heads Tails 


Heads 
P, Selections 
'Tails 


! Here we include appropriate ''stop rules" in the conventions of the games. 
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where at the end of a play P» pays Pi the amount corresponding to the 
selections. We can further summarize the problem by means of the 


matrix 


1 —1 

nn) 

Here the rows correspond to the possible choices for P; and the columns 
to the possible choices for P». As soon as P selects a row and P» selects 
a column, the game has been played, and the pay-off value at the inter- 
section of the selected row and column is paid to P;. 

After P, makes his selection, he must be careful to conceal his choice 
from Ps; otherwise P; will select the column that will yield a pay-off of 
—]1. P;can hide his selection by using some chance device. For exam- 
ple, he could place two pieces of paper in a hat, one slip marked with an 
H and the other with a T. Before each play P; removes one of the 
marked slips from the hat, choosing heads if the slip has the H or tails if 
the slip has the T. "The probability of P; choosing heads is 15, and the 
probability of choosing tails 1s also l5. Let us assume P, makes his 
choices in this manner. If P; selects heads, then P, has a mathematical 
expectation (1.e., the sum of the quantities obtained by multiplying the 
probability of choosing each row by the corresponding pay-off) of 


1$0) 4 3(-1) 20 
and 1f P» selects tails, 
(C71) 4 1) 2 0 


Hence, P;'s expectation is zero. If fact, this is the only mode of play 
for P, that does not expose him to a negative expectation. "To show this, 
suppose P, plays heads with probability zr and tails with probability 
l-—z,where 0 « r- 1. If P; plays heads, Ps expectation 1s glven by 


Eg —z(1)-d-(1—2)(-1) 2-2: — 1 

and if P, plays tails, P;/'s expectation is 
E,;-2z(—1)4-(0—2)10)21-—2z 

For0 € z « 15, Eg « 0; and for ló«z €1,E,-«90. Hence P, must 


select heads and tails with equal probabilities to obtain a zero expecta- 
tion. The same situation is true for P;. 
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Let us generalize the notions of the above discussion with the 
following definitions: 


Definition 1l. A matriz game T 1s determined by any m X n matrix 


O11 Q12 Q1n 
Q»1 | Q9 Q», 

RERO EN DN IE (aij) 
Om1 G2 Gmn 


in which the elements a;; are any real numbers. "The matrix A is 
termed the pay-off matrix. "The element a;; represents the amount 
paid to P, by Ps1if P, selects the move associated with the zth row 
and P; selects the move associated with the jth column. 


Definition 2. By a mired strategy for P, we shall mean a row vector 


X — (xis, . . . ,z4) of nonnegative numbers z; such that 
Pic pet noue cd 
By a mired strategy for Ps we shall mean à column vector 
Y — (yyys, . . . yx) of nonnegative numbers y; such that 
ux un cos epe qase 


'The elements r; and y; represent, respectively, the frequencies 
with which P, selects his ith move (row) and P: selects his jth move 
(column). For the matcehing-pennies game, a mixed strategy, i.e., 
the probabilities for selecting heads or tails, for P, could, for example, 
be any of the following: (1,0), (0,1), (14,34), (l5,15). Ps could 
employ similar mixed strategies. 


Definition 3. For each  — 1, 2, . . . , m, the mixed strategy which 1s 
l in the zth component and 0 elsewhere is called the «th pure strategy 
for P, and will be denoted by 7. 
Similarly, the jth pure strategy for P», denoted by j, 1s à mixed 
strategy for P» which is 1 in the jth conponent and O elsewhere. 


Let us next consider the matrix game determined by the pay-off 
matrix 
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If P, chooses any pure strategy ?, he is sure of being paid at least min aij. 
7 


For example, if the matrix game has the following 3 X 4 pay-off matrix: 


D-—5 0 
zu MES 
4 2 —1 0 
we have 
min ài; — 443 — O min d»; — ds9 — 1 
j j CI-D 
min da; — 433 — —1 


J 


Since P, can select any 7, he can, or course, select the ? which maximizes 
his gain. "That is, he can select the pure strategy that will make min qi; 


J 
as large as possible. By using this pure strategy, P; can win at least 


max min qj. For the 3 X 4 pay-off matrix we have from (1.1) that 
D j 
max min dj — d» — l 
D 77 


If P; selects a j, the worst thing that can happen to him is to lose 
max à. JP»can then select the pure strategy which minimizes his losses. 


1 
By using this pure strategy, P» can keep P, from winning more than 
min max aj. For the 3 X 4 matrix we have 


9j 1 
max Qj; — 03-4 InXU S -- 9 mE) 
i i 
màX Qj — 033 — à IDAX Qj — d14- 4 
i i 
and 
min max Qi; — dsa — 3 
j D 
If 
max min dij; — min max Qj; — v (1.2) 
D 7] Jj D 


then P, ean be sure of winning v, and he can be prevented from winning 
more than v by P». Any matrix game for which (1.2) holds can best be 
played by P, and P; playing the corresponding pure strategles. Since 
any deviation from this strategy by P; lowers his opportunity for winning 
at least v and since any deviation from this strategy by P; raises Pj's 
opportunity for winning more than v, these pure strategies are called 
optimal pure strategies. 
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The following matrix game is one which has for its optimal pure 
strategies X — (1,0,0) and Y — (1,0,0): 


Sr ug $5 

Lo m 

(Qr si 
Here 


max min d — min max Q, — d41-— 3 
2 7 J 1 

'The reader will note that the element a;;is both the minimum element 
in its row and the maximum element in its column. Any such element is 
called a saddle point. f the element aj; 1s a saddle point, then the pure 
strategies k and | for P, and P5, respectively, are optimal pure strategies 
and ag — v. 

Since not all matrix games can be optimally played by means of pure 
strategies (e.g., matching pennies), we need to consider the notion of 
optimal mixed strategies. 


Definition 4. "The pay-off function for P, 1.e., the mathematical expecta- 
tion of P, is defined to be 
E(X,Y) -— XAY — Y D. X1Qigy; 
iz1j-1 
where X — (x25, .. . £4) and Y — (ys,ys, . . . ,y«) are any mixed 
strategies for P, and Ps, respectively. 


To illustrate this definition, let the matrix game have the following 
pay-off matrix: 


0r TF 1 
1 di 
—1 Il 0 


If P, selects the mixed strategy X — (xi,25,23) and P; the mixed strategy 
Y — (yyys,y3), then E(X,Y) is equal to the matrix product 
0 —1 IN f ya 
(23,225,753) 1 0 —1]l ye 
—1 il 0/ Nys 
Or 
EY) epp py d C2 z3)ys d- (1i — 2)ys 
If X — (0.1,0.4,0.5) and Y — (0.3,0.3,0.4), then 


E(X,Y) - —0.03 
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Hence if P, and P; employ the above mixed strategies, P;'s expected pay- 
off is —0.03 of a unit. 


Definition 5. .À solution to a matrix game T' is a pair of mixed strategies 
Xm E 2) 
Y ugs «o yd) 
and a real number v such that 
E(X3) for the pure strategies j — 1,2, .. . ,n 
E(àY) € v for the pure strategies ? — 1, 2, .. . ,m 


The X and Y are called optimal strategies, and the number v is called 
the value of the game. 


From this definition we see that, if P, selects his moves as determined 
by the probabilities of his optimal strategy, then no matter what moves P» 
selects, P, can expect to win at least v units. .À similar statement is true 
for P; i.e., he can expect to lose no more than v units. We should note 
that v ean be positive, negative, or zero. For the game of matching pen- 
nies, whose pay-off matrix is 


1 —l1 

e 
the value of the game is 0 and the optimal strategies are X — (15,14) 
and Y — (15,14). 

The following game from Kuhn [67], the skin game, is an example of a 
game which at first glance appears to be fair, i.e., has a value of v ^ O0. 
The two players are each provided with an ace of diamonds and an ace 
of clubs. .P;is also given the two of diamonds and P; the two of clubs. 
In the first move, P, shows one of his cards, and P5, ignorant of P,'s 
choice, shows one of his cards. P, wins if the suits match, and P; 
wins if they do not. "The amount of the pay-off is the numerical value of 
the eard shown by the winner. 1f the two deuces are shown, the pay-off 
is zero. "The pay-off matrix is then 


€ ^ 24 

$ ]. 1T 2 
*|-—1 1 l 
29 2. -1 0 


Since every element of the third row is greater than or equal to the corre- 
sponding element in the first row, there is no advantage to P, in playing 
his first strategy. We can then assume he will always play his third pure 
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strategy in preference to his first, and hence z; — 0. Wesay that row 1 is 
dominated by row 3, and we can eliminate row 1 from our pay-off matrix. 
We now have 


—1 I 

2s c] 0 
Ps ean apply the same type of analysis to the reduced pay-off matrix, and 
since the elements of the second column are less than or equal to the cor- 


responding elements of the third column, P; will not select his third pure 
strategy; that is, y — 0. "The final reduced pay-off matrix is 


ue 


In looking for a solution to the game defined by this reduced pay-off 
matrix, we should first determine whether there 1s a saddle point. Here 
there is none. If P, selects his second strategy with probability r and 
his third with probability 1 — x and if Ps uses probability y for his first 
strategy and 1 — y for his second, then 


EQGY) e (—922-2)y 4 Qx -— TXl- y) 


The reader can readily verify that an optimal solution to the original game 
is given by X — (0,365,242), Y — (26,26,0), and E(X, Y) — v » 1$. Hence 
the game is biased to P,. "The reader can also verify that, for any other 
possible mixed strategies, X — (0, z, 1 — xz) and Y - (y, 1 — y, 0), 


WOCOTYP ORO Y) -—EG Y) 
Or 

max min E(X,Y) — min max E(X,Y) — vf 

x- ow Y ox 

In other words, if P, plays his optimal mixed strategy, his value of the 
game will not be less than E(X,Y), no matter what strategy Y is used by 
P, Similarly, if P; plays his optimal mixed strategy, he can expect to 
keep P; from winning more than E(X,Y). If both play their optimal 
strategies, then the expected outcome can be predetermined. This mode 
of playing is a conservative one, in that it does not let the players take 
advantage of any deviations made from the optimal strategy by their 
opponents. 


Definition 6. .À symmetric game has a skew-symmetric matrix; that is, 
Qj — —Qj. 
t See Kuhn [67] for derivations of these expressions. The first is called the saddle- 


point statement of the theory of games, while the second is the corresponding min- 
max statement by von Neumann and Morgenstern. 
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We next show that the value of a symmetric game is zero and that 
both players have identical optimal strategies. 'The pay-off function for 
P,is given by 


TT TL 


E(X,Y) —- XAY — »» » 21043; 


1—173-1 
It is easy to verify, for a skew-symmetric A and for X — Y, that 
E(X,Y) 2 XAX — 0 


Hence, if both players use the same mixed strategy, then the expected 
valueiszero. Let the respective optimum strategies be denoted by X and 


Y. We have max min XAY — min XAY — v. If P; uses any mixed 
Xe y Y 


strategy, then XAY 7» v. But we know, for Y — X, that XAX - O0. 
Hence, since P; can always make the value of the game no greater than 
zero by playing Y ^ X, we must conclude thatv «X 0. Similarly, we have 


m max XAY — inse XAY — v. If P, uses any mixed strategy, then 
x 


XAT «v. ForX - "Y we have YAY — 0, and hence P; can always make 
v 2 0. "Therefore, we must have v — 0, and both players have the same 
optimal strategies X — Y. 

As an example of a symmetric game, let us develop the game matrix 
for the children's game of Stone, Paper, Scissors. In this zero-sum two- 
person game both players simultaneously call out stone, paper, or scissors. 
The combination of paper and stone is a win of one unit for the player 
calling paper (paper covers stone); stone and seissors 1s à win for stone 
(stone breaks scissors) ; and scissors and paper is a win for scissors (scissors 
euts paper). A call of the same item represents no pay-off. "The pay-off 
matrix in terms of P, as the maximizing player is 


Stone Paper Seissors 


Stone 0 —1 1 
Paper i 0 -1 
Scissors V —1 ll 0 


The optimal strategy for both players is (14,14,14). 

In the next section we shall want to change the value v of a game by 
adding a fixed amount w to each element of the pay-off matrix (aj). In 
this connection we wish to show that, for the new pay-off matrix (a; -- w), 


the optimal strategies are the same as for the original game and the value 
of the new game is v 4- qw. 


By definition we have 


Tn "n 


E XY)- Y Y za; (1.3) 


1-17-21 
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for the original game and 


TIL 


EQXY)- Y y vai d- w)y; (1.4) 


1—-17j-1 
for the modified game. . Expanding (1.4), we have 


7h 


Es(X,Y) BY » v;Qgy; Fw Y S ty; (1.5) 


1—-17j-21 7—13-1 


Since Zz, — Zy; — 1, (1.5) becomes 


Th 


Es(X,Y) - » x T;0;5Uj apu 


1-17j-1 
From (1.3) we have 
E4(X,Y) — E,(X,Y) - wv 


We see that the constant w does not play any role in the selection of an 
optimal strategy for the transformed game. In fact, the second game is 
the same as the original if, before the start of each play, P» gives P, w 
units. Hence the optimum strategies are the same, and 


E;X,Y) —- E,X,Y)--w-v-rv 


By selecting à suitable w we can make all elements of the pay-off matrix 
positive, thereby ensuring that the modified game has a positive value. 
We next state, without proof, the following theorem: 


The fundamental theorem of matrix games. For every matrix game 
max min Z(X,Y) and min max E(X,Y) exist and are equal. That is, 
X Y Y x 
every matrix game has a solution. 
' The reader is referred to Kuhn [67], McKinsey [72], Luce and 


Raiffa [71a], von Neumann and Morgenstern [80], and Owen [85aa] for 
proofs of this theorem. 


2. THE EQUIVALENCE OF THE MATRIX GAME AND THE 
PROBLEM OF LINEAR PROGRAMMING 


We shall assume that we are given an arbitrary matrix game: 
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From Definitions 4 and 5, the problem for P; is to find a vector 
X c Ppdg e eus sm) 


and a number v» such that 


Gif d- dsiX3 d- ^ coc oB Os zu 

QE Uie: JEU 

dista cb ast oto BR aUe cU 
Z14d- sd 0-4 darse 1 (2:1) 

X1 L 0 

X2 2 0 

Do 20) 

Similarly, for P» we have 

01317103202 5 2 ose aal sn D 

aoi 4- Q221/2 zip S DO ES SU EET SU 

(iUi ch malo de o SEWanUne S D 

Wrcb o Juscbote 4e Wave 
UE 20 (2:2) 

y2 zz 

ys 2 0 


Since every element of A can be made positive by the addition of a suitable 
constant to all the a;;, we can assume that v » 0. Let us divide each of 
the relationships in (2.1) and (2.2) by v and let 

! AE 


i Ü 


rT,-— and y 


Note that 
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and 


e [re 


»E -iyy- 
J 


z 
Dovg rur; / 5 B . 
Hence, by minimizing ». rz, P, will maximize the value of the game, and 
1 
by maximizing » y; P» will minimize the value of the game. We can 


J 
then restate (2.1) and (2.2) in terms of equivalent linear-programming 
problems and obtain the following symmetrie dual problems: 


The primal problem. Find & vector X' — (zxj,25, . . . ,3,,) which 
minimizes 
Dr 
subject to 
Qux, -F dax, occ d o084,21 
Qixtj JF Gaxx, -b c coc db aom, 2l 
0E, Lodi. o de dum, c 
2:20 
The dual problem. Find a vector Y' — (yy, . . . ,y,) which 
maximizes 
gcgaupeb ot Sep. 
subject to 
Qiii Jr 1215 EOM; 2o dtr OV, Su 
dij, d- dex; P cocco d asy, El 
Oni, d- Om3ys -- ds. X 1 
y; 20 


Since every game has a solution, optimum solutions to the above problems 
exist and 
IT 


min Y a; -— max 5 yj E 
D j 


It should be noted that, if only the primal or only the dual is solved, the 
optimal strategy for the other problem is contained in the simplex tableau 


1 The set of z; and y; which satisfies the linear-programming problems must, of 
course, be converted to the optimal z; and y; that solve the game problems. 
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of the corresponding final solution. "The optimal strategy corresponds to 
the z; — c; elements for the slack vectors. 

An alternate method for reducing the game problem to a linear- 
programming problem is the following: For P; the problem 1s again given 
by (2.1). Let us write the first n inequalities as equalities by subtracting 
a nonnegative variable from each to obtain 


QiiXi d- deis b coco dR Omitm — Ymjaà iU 

032253 1-102299. 91 0:8 SIE Omas — Xm42 E 

Q1nX31 4 O22 4 egi ou 4j Um nm — mm -U 
piste DO d quo SIM cis Um - 

4 2 0 


By subtracting the first equation from the succeeding » — 1 equations 
and using the first equation for the objective function, since it 1s equal to 
v, we then have the equivalent linear-programming problem of maximizing 


Qixti d GayXa dE co coc c OuiXm — Xmai c— V 
subject to 
(a12 y Q11)21 E (a9 E 251)T2 Ic XE (am s E-sancsess TIm-1 — XZma2 iU) 
(Q1 ms d1)21 sim (Q5 A 051)T2 * ey - (o. zwi (n1) m c Tmal YTm-n 0 
zi gucp esc Ey -1 
Ti m 0 


Example. Here we are given a matrix game with the following payoff 


matrix : 
3 —2 —4 
ÀA-|-1 4 2 
2 2 6 


The development of the corresponding linear-programming problem 

(using the above alternate method) is as follows: The initial system of 
constraints is 

91; — dX. -- 2-3 

— 2244 Eg 45 1 2m3 

—4z 4- 2x» 4- 62; 

Xi-F- X. v3 


X1 


Cy dv 


e 
2 
IV. IV. IV 
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Subtracting nonnegative slack variables, we have 


OXi — X3» 223 — x4 zy 
—2z, -- Av» 4- 223 — c5 zm 
— 4a, -F 2x5 4- 6x3 — 3&7 

Gi cb- d$» oda —-1 
X 2 0 


By subtracting the first equation from the second and third, we find 
that the corresponding linear-programming problem is to maximize 


3zi— zx.-d-2x3— x4 
subject to 
—baxi -- (BUE Jj 24 LUE zu 
— [24 4- 3v» -- Axa 4-24 — 216 — 0 
doi mp" ES Su dus —-1 
Xi 2 0 


As the computational techniques for the direct solution of large matrix 
games are rather cumbersome, the efficient computational techniques of 
linear programming are usually employed to solve such games. There 
are, however, rapid iterative methods for solving games that yield good 
approximations to the optimal solution (Williams [101] and Luce and 
Raiffa [71a]). 

'The converse problem of expressing a given linear-programming prob- 
lem as a matrix game can be readily accomplished by considering the 
problem and its corresponding dual. Let the original problem be given in 
terms of the inequalities 


ndr 79 3 07v bi 
SESS ER E. MEM (2:9) 
utu: c rase Dosis Das 
DIDI $e eger (2.4) 


where z; 7 0 and f is the minimum of the objective function (2.4). "The 
associated dual is 


ü0pe- "o PE ritis S 01 
noon m TT (2.5) 


s 
bud obo. Sg (2.6) 
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where w; » 0 and g is the maximum of the objective function (2.6). 
From the duality theorem of Chap. 5 we have that, if a finite optimum 
solution to either problem exists, then a finite optimum solution for the 
other also exists and f — g. Let us assume that such solutions do exist 
for the problems (2.3), (2.4) and (2.5), (2.6). Let (xj . . . za and 
Qo, . . . jw») be solutions to the former and latter problems, respec- 
tively. Multiplying the first inequality of (2.3) by wi, the second by ws, 
ete., and summing the m inequalities, we have 


byw, ea CMPSS J- bs «x e 010) 21 -r fot — 3 dii) Xn 
md i-1 
(2.7) 


From (2.5) we can replace the sums in (2.7) by their corresponding c; 
to obtain the following inequality that is true for all solutions to the 
primal and dual: 


bw, EH ON EN 4 Das E C111 m bel: zs Qe (2.8) 


This implies that g € f. 
The reverse inequality of (2.8) 1s given by 


Duy eos sore uda e oup E PEU 
Or 
buy E scscoE Da c (ever scored t (2.9) 


This inequality restricts g 2 f. Hence, à simultaneous solution to the 
complete set of inequalities (2.3), (2.5), and (2.9) will yield optimum 
solutions for the primal and dual with, of course, f — g. 

Let us multiply each inequahty of the set (2.3), (2.5), and (2.9) by an 
unknown positive variable z to obtain 


aac tob RE. 015 7-0 
Qo di uis 2g ar 4 rm UT bz p 0 

rina s o e aytba) Tog 0 ÍuI0) 
- ub se sen espetdifa) ctoc2L0 
(by, ls e gale SE ba04) m (eda -- "Qror do 4 60) E 0 


where d; — zr; and iw; — zw;. A^ solution to (2.10) with z » 0 will yield 
a solution to the original primal and dual problems. "The above has the 


f The inequality (2.8) is equivalent to formula (1.13) of Chap. 5. 
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following representation in terms of matrices: 


Oo A' —c' 
(XWez)| —A Oo b. 4 (0) (2:10) 
c —b O 


Here A is an m X n matrix; b, c, X, and W are row vectors; and the O 
matriees are of the right dimensions. We can consider the skew-sym- 
metrie matrix of (2.11) as defining a zero-sum two-person symmetric 
game and the row vector (XWz) as the strategy vector for the maximizing 
player P,. Since the value of a symmetrie game is zero, we have from 
Definition 5 that a solution to inequalities (2.11) with 


23; $9,4-2 1 


is a solution to the corresponding game. By the fundamental theorem, a 
solution to this game always exists. However, the associated linear- 
programming problems may not have any solutions. "This will be the 
case if the game solution yields an optimum strategy with z — 0. Hence 
a game solution with 2 7 0 will yield optimal solutions to the primal and 
dual linear-programming problems as defined by (2.3), (2.5), and (2.9), 
where z; — i;/2 and w; — wi/z are the respective optimal solutions. 

As an example of parametric programming techniques applied to the 
theory of games, let us next consider the matrix game with the following 
pay-off matrix: 


À À À 
Q»1 022 Q2n 
Gm1 [47707 y fe Gmn 


The parameter A can assume any real value and represents the pay-off to 
player one (the maximizing player) when he employs his first strategy. 


Player one wishes to determine a mixed strategy X — (xis, . . . ,vw) 
such that 

AT15p deyra eee Eri. 2 

Aqu deiüssua oe tom Eus c9 

Aqu ous c perra a zr 

queso Xew iecur pes 1 


where z; 2 0 and v, the value of the game, is to be a maximum. 
Following the discussion above, the equivalent linear-programming 
problem is to maximize 


Azi 4 (EXE EDS OO re GQmiXm — Xmjil UU 
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subject to 
(29 — 8:1)Es ^F - : 7 P (Uma — Qmi)Em d Xmai — Yme x 
(Qa. — Q21)3» P - : * P (Ama — Ami)Zm d Emaài — Zm4n — 0 
X1 -F X9 Es UOS 4 Tm — 
where c, — Oforz —1,2,...,m-rm. 


This parametric linear-programming problem can be solved for all 
possible values of the parameter 4 by the procedure described in Chap.8. 
Games which have more than one parametrized row, e.g., 


»X X EIS » 
Hu Hu EE Hu 
Qai  Q3a2 vs 8n 
USC UR (0 o RECS DUEMUT E ror 


can also be transformed into equivalent linear-programming problems. 
However, efficient computational techniques for solving problems with 
more than one parameter are not available. For a discussion concerning 
the two-parameter objective function, see Gass and Saaty [47]. 


REMARKS 


'Theintroductory material on the theory of games is from Kuhn [67] and McKinsey 
[72]. The material in Sec. Z is from Dantzig [19]. For additional reading the reader 
ls referred to von Neumann and Morgenstern [80], Luce and Raiffa [71a], Owen [85aa], 
and Williams [101]. 


EXERCISES 
1. Show that 


max min aj; X min max di; 
i j i [ 


in the arbitrary matrix 


Q1 Q12 Q1n 
Q21 Q22 Q2n 
Om1 m2 Wo E mus Gmn 


2. Write the pay-off matrix and the equivalent linear-programming problem to 
the following game: Each player selects an integer from the set of integers (1,2,3). 
The player with the lower number wins 2 points unless his number is 1 lower, in which 
case he loses 4 points. When the numbers are equal, there is no pay-off. 
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3. Transform the following matrix games into their corresponding primal and 
dual linear-programming problems: 


oe) 
TOO ED 


QE 1 
1 Td 
itc essit 0 


4. Write out the associated game matrix for the following linear-programming 
problems: Minimize 


X2 — T3 
subject to 
Zi-- s — 3 0 
—zi-F zs 4-323; 1 
z;1— 925 -- x; X 1 
maet) 
Maximize 
wi ws- ws 
subject to 
wi--4ws»--3wi € 1 
4wi| — 4w» 4-2ws 1 
3wi — ws -d-5ws X 1 
w;ao 


b. Solve the following parametric game for all values of the parameter: 


À DEN 
A-|[-1 4 0 
S» sull 


6. By letting v — v1 — vs, that is, the difference of two nonnegative variables, in 
Eqs. (2.1), determine a third linear-programming formulation of the matrix game. 

7. Solve the numerical example of Sec. 2. 

8. Develop a graphical procedure for solving any 2 X n matrix game. ([HiwT: 
Let v be one variable and z; the other variable of the two-dimensional graph, with 
zx» — 1l — z,, and use the formulation of Eqs. (2.1).] 

9. Solve the skin game by the graphical method of Exercise 8. 


PART FOUR / NONLINEAR 
PROGRAMMING 
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CHAPTER 13 / NONLINEAR 
PROGRAMMING 


The linear-programming model, along with the implied concept of viewing 
complex systems in terms of interrelated aetivities,! has proved to be a 
major contribution to the field of scientifie decision-making. | Applications 
of linear programming exist in every category of industrial, business, and 
government endeavors. "Phe versatility and adaptability of this decep- 
tively simple mathematical model appears to have no bounds. However, 
early workers in the field recognized certain implied limitations of the 
linear model and began to develop theoretical, as well as computational, 
foundations for special and general nonlinear-programming problems.? 
One can, for example, trace such developments by studying the growth 
of programming applications within the petroleum industry (Charnes, 
Cooper, and Mellon [13], Manne [73a], Symonds [93a], Garvin et al [455] 
and Catcehpole [8d]). "This industrial area yielded some of the first 
successful nonmilitary applications of linear programming. Extensions 
of these applications led to the need and development of many nonlinear- 
programming techniques. Nonlinearities in the programming model of 
an oll refinery arise 1n both the objective function and the system of con- 
straints. As Catcehpole [8d] and others point out, the cost of crude oil 
may inerease with the quantity required, while the use of lead as an 
octane improver has a decreasing effect as more lead is used; i.e., the 
mathematical relations which describe these statements are not linear. 
The consideration of these and other nonlinearities has advanced our 
ability to develop mathematical models of complex systems. We can 
now describe and deal with many generalizations of the linear model that 


! See Dantzig [16c], Chap. 3, for a detailed discussion along these lines. 

?'The reader is referred to the work of, among others, Arrow, Hurwicz, and 
Uzawa [3], Barankin and Dorfman [3d], Beale [5e], Charnes and Lemke [15], Dantzig 
(16c], Dennis [32/), John (62c], Kuhn and Tucker [69], Markowitz [74d], Slater [90d], 
and Wolfe [1054]. 
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take us into the field of nonlinear programming—or into the more inclusive 
field of mathematical programmang.! 

'The purpose of this chapter is to present selected basic material 
necessary to the understanding of the foundations of nonlinear program- 
ming.? "The material requires a broader mathematical background than 
does that covered in previous chapters. However, except for basic notions 
from the ealeulus, this chapter is essentially self-contained, assuming 
knowledge of the preceding discussions. 


1. THE GENERAL PROBLEM OF MATHEMATICAL PROGRAMMING 


As noted in Sec. 1 of Chap. 1, the mathematical model of the linear- 
programming problem ean be described using relationships of the form 


Gyzy d day d- coco dp agmd- ccc o6. — 


From a mathematical point of view this is a rather restrictive relationship 
in that it excludes a wide variety of possible interactions between the 
variables. For example, the output of an activity might be functionally 
related in a nonlinear fashion to the output of other activities or even to 
itself. Nonlinear expressions of the form 


g(X) E g (1,22, gre 22 — QT? M 31215 4- ris 4J- Q404* -—h 


and trigonometrie, logarithmic, and other special functional forms can 
occur in the constraint set and objective function of practical, as well as 
theoretical, problems. Other nonlinearities include the restriction of a 
variable to integer values; for example, z; — 0,1,2,3, . . . , as diseussed 
in Chap. 9. 

For our purposes, we define the problem of mathematical program- 
ming to be: 

Find a vector X — (xis, . . . ,2,) which minimizes (maximizes) the 
objective function f(X) subject to 


gx) s 0 fopr ed V^ om (1.1) 


Here f(X) and the g;(X) can be considered to be general functions of the 
variables zi, x, . . . , x4. If a problem contains equality constraints or 
inequality eonstraints going in the opposite direction, these constraints can 


! Bee the books of Abadie [1], Boot [6c], Saaty and Bram [88b], Hadley [52d], 
Kunzi and Krelle [69a], Vajda [97a], Zoutendijk [108], Fiaceo and MeCormick [42c], 
and the survey article by Dorn [40a]. 

? 'The reader interested in specific applications should consult the references given 
above and Bracken and MeCormick [7a]. Although apparently not as plentiful and 
pervasive as linear programming, nonlinear applications include chemical equilibrium, 
least-square and Chebyshev approximations, stock portfolio selection, and many 
problems from mathematics and economies. 
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be expressed in the form (1.1), so there is no loss of generality in assuming 
the constraints to be of the above form. 
For a linear-programming problem, problem (1.1) becomes: Minimize 


fK(X)-on-ctccc-d eu 


subject to 


gXX) — auzi-d cc: d osr.— 0,x0 
Q5 (X) Gm1X1 *E PS als GmndXn — b, S 0 
Qm(X) — xi 20 
Qm (X) — £. 0 


A simple example of à nonlinear problem involving logarithmiec functions 
in the objective function (Dorn [40c]) 1s as follows: Minimize 


f(xiz3) — —log zi — log zs 


subject to 


xix. 2 
X1 2-20 
103 220) 


The graph of the constraint set and the objective function, for various 
values of the objective funetion, is shown in Fig. 13.1. "The solution 
space is the shaded triangle and the optimum occurs at the point (1,1), 


FIGURE 13.1 
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not at an extreme point of the solution space. When the constraint set 
can contain nonlinear functions, we are not necessarily limited to the 
nice mathematical situation found in linear programming, i.e., the search 
for an optimizing extreme point of a convex solution space. "The mathe- 
matical-programming problem could have a nonconvex solution space, 
an interior optimum solution, and other characteristics which compound 
both theoretical and computational developments. 

One of the main difficulties encountered in the realm of mathematical 
programming is the determination of a solution point which appears not 
only to optimize the objective function at that point, but to optimize the 
function over the complete range of the solution space. More precisely, 
we have: 


Definition 1. A function f(X) has a global minimum at a point X? of a set 
of points K if and only if f(X?) € f(X) for all X in K. 


Definition 2. .A function f(X) has a local minimum at à point X? of a set 
of points K if and only if there exists a positive number e such that 
f(X?) € f(X) for all X in K at which |X? — X|| « «e. 


In one dimension, it is easy to illustrate local and global minima for a 
function f(X) as in Fig. 13.2. Here K is the set of all nonnegative z. 
The points z — 0 and z — zr» are local minima, while x — z,; is the global 
minimum. (A global minimum is also a local minimum.) The reader 
should note that global and local maxima ean also be defined by obvious 
changes in Definitions 1 and 2. 

Although nonlinear optimization problems call for the determination 
of a global minimum, computational procedures will, in general, lead to 
a solution which is only a local minimum. Moreover, it is usually 
impossible to determine if a local minimum is really a global minimum. 
Even if this could be done, computational procedures have no way of 


i KG) 
f(0) 
if x) 
Ga) | 
T ——————— 
(0, 0) X1 X2 * 


FIGURE 13.2 
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proceeding from a loeal minimum to a global minimum. This is not the 
case, of course, for the linear-programming problem. For that problem, 
we can show that the simplex algorithm arrives at a solution point which 
is not only a local minimum, but also a global minimum. This is a key 
charaeteristie of most computationally traetable nonlinear-programming 
problems. Thus, our discussions below will be directed toward problems 
for which we ean demonstrate mathematieally the coincidence of the 
local and global minima. To accomplish this end, we next deseribe and 
develop some necessary mathematical notions. 


2. MATHEMATICAL BACKGROUND 


a. Quadratic forms. In some mathematical-programming problems, 
the objective function is restricted to a specialized expression, a positive- 
definite quadratic form. 


Definition 3. .A quadratic form f(X) 1n n variables 1s an expression of 


Lu n 
second degree which can be written as f(X) — 9 » Cor OP 
q-—0134-1 
fX) s Oudada ts OIATMPS cp sce ecce qub. 
» M -F Cni 21 AF E 3b CnnX nn 


For » — 2 we have 
; 2 
f G,22) -— Cuv; -F CisziXa d- €oiXiX2 d Cost 


If C — (c), then the quadratic form is expressed as f(X) — X'CX, where 
X cr. ...,5918an0« X1 eolunnu veetor and Cis ana X ^ matrix. 
born 2, 


, 
X MH * C11. C12 21 
goes ee cre oui) ) 
T2 X2 C21 C22 T2, 
31 
e (C111 seauEs. ys -T- C231) d 
2 
— £X -- Cis*iX» P CoXaxi d- Cos 
2 
z- Cii in (C12 3E €21)X122 -F C221» 


Noting that for any X and C the product X'CX is a scalar and thus 
X'CX — (X'CX) 2 (OX) (X — X'C'X, we can wnte 


X' (Cu cox 


XX cU XR Xcx- ; 
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'The matrix 14(C -4- C^) is a symmetric matrix with the general element 
Ló(cg -- cj). In this form, the coefficient of zr; 1s Lló(c; 4- cj) and of 
z;c;ls lg(cg; d- c). However, for the total product term of z; and zj, the 
coefficient is (ci; - cj), as it is in the original form X'CX. Forany C we 
can redefine the coefficients of z;z; to be L6(ci; -- cj) and of zz; to also be 
Ll6(ci J- cj) so that the matrix representation of the quadratic form 1s in 
terms of a symmetric matrix. In what follows, we shall always assume 
that the matrix C is symmetric. 


Definition 4. A quadratic form is called positive definite if X'CX 2 0 for 
X x0. Forexample f(X) — xj 4- 2? is positive definite. 


Definition 5. A quadratic form is called positive semidefinite if X'CX — 0 
for all X and there exists atleast one vectorX zt Osuchthat X'CX — 0. 
'The quadratic form 


JOD x]. — muraux: 3 - 
(zi E- qud 2-0 


for all z; and z» and equals zero for z; — z; — 1, and thus f(X) is 
positive semidefinite. 

Negative defimite and negate semidefinite forms can also be 
defined by appropriate reversal of the inequality signs in the above 
definitions. 


For quadratie forms of small dimension we can test for positive 
definiteness by the following theorem (Hadley [520]) : 


Theorem 1. Te quadratic form f(X) — X'CX is positive definite if and 
only 1f the principal minors, i.e., the minors obtained by taking succes- 
sively larger minors along the principal diagonal (see below), are all 
strictly positive. 'Thus for n 2 4 we must have 


Cii C12 Cia 
20 C31 €22 0:9 2 0 |C| »- 0 


C31 C32 Cas 


€1 2 0 


b. Convex functions. 'The main mathematical concept required in our 
development of nonlinear programming is that of a convex function. 
Given that the constramts g;(X) and the objective funetion f(X) are 
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convex functions we can, as shall be shown below, prove that a local 
minimum is also a global minimum. 


Definition 6. .À function f(X) — f(xi,zs, . . . ,z,) defined over a set of 
points which lie in a convex set K in E, is called a convez function if 
for any two points X, and X; in K and any 0 € ^ € 1 we have 


fX; t (1 — X)X) S NU) o (0 — f(X3) (2.1) 


If strict inequality holds in (2.1), f(X) is said to be strictly convex. 
Similarly, a function f(X) is a concave function if (2.1) holds with the 
inequality reversed, that is, —f(X) 1s convex. 


A convex function f(X) in one variable is pictured in Fig. 13.3. The 
reader will note that the expression Af(zi) 4- (1 — X) f(zs) 1s the ordinate 
corresponding to an abscissa of Ax; -- (1 — X) z; for the line segment 
joining the points [z;, f(z1)] and (zs, f(x?)]. "Thus, a function 1s convex if 
every point on each such line segment overestimates the corresponding 
point on the function; i.e., the line segment lies above the function. A 
linear function is both convex and concave; the sum of convex functions 
is also a convex function;i.e.,iff;(X),? — 1, . . . , kareconvex functions, 


then f(X) — Y A0) is convex. 


1-1 


Theorem 2. The positive semidefinite quadratic form f(X) — X'CX is a 
convex function for all X in E,. 


Proof. We wish to show for 0 € ^ € 1 and all Xi, X; in E, that 


fX, -- (1 — XX] € N(X)) 4-1. — Xf(X3) 
or 


fX, -- (0 — X)Xj — N(X:)) — 0 — Xf(X;)) € 0 (2.2) 


(0, 0) 
FIGURE 13.3 
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Noting that XCX; — XjCX,, the left-hand side of (2.2) can be 
rewritten as 
[AX; -- (1 — X)X4/CDAXs 47 (1. — X)Xi] ; 
— AXjCX, — (1 — J)X;CX; 
- XX/CX; 4- (1 — AyX;CX; 4 2A(0 — X)X;CX, 
—AX(OXQ— (1 -*965€X; 


- (4? — 3)XICX; 4- (1 — XJ. — 3)X; CX, — X;CX;] 
J- 24(1 — X)XjCX; 


- AQ — DXI|CX, -- A(4 — D)XjCXs — 240 — 1)X; CX» 
—- A( — LD)[Xj|CX; 4- X;CX, — 2XjCX;] 
— AQ — D[(X: — X3)'C(Xi — X5)] (2.3) 


Sinee the form X'CX » 0 for any X, for (2.3) we have 
[X —X3'/C(X, — X,)] — 0. Biuee A( — 1) «O0 for Ü A «d 
and A(À — 1) 2 0 for — O or 1, then (2.3) will always be less than 
or equal to zero for all vectors X; and X», which was to be shown. 
Thus f(X) 2 X'CX is convex for all X in E,. 


Theorem 3. f f(X) is convex on a convex set K, then f(X) has at most one 
local minimum. — If there is such a minimum, it is a global minimum 
and 1s attained. on a convez set. 


Proof. Suppose there is a local minimum at a point X. Forany X —^ X 
in K, by definition of a local minimum and convexity of f(X), we have 


f(X*) € f((1 — X)X? 4- AX] € (1 — )f(X9) - Af(X) (2.4) 


for ^ a sufficiently small positive number. From the extremes of 
(2.4) we have 

fO s. -—Oof0) 4 X005 

Af(X*) € Xf(X) 
and since À » 0, then f(X^) € f(X), which implies f(X?) is a global 
minimum by definition, as X is any point in K. 


If X^ and X! are two points at which f(X) attains its minimum 
value 2,, then for0 X A € I1 


£« S f[(. — X)X? -- AX!] S (E — (X9 -F Xf) 9 


Hence, f(X) also attains its minimum at X — (1 — A)X? 4- AX! and 
thus the set of solutions is convex. 
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Theorem 4. Let K be the set of points of E, which satisfy the constraints 
TC QURE C E NET 
X 0 


If the gi(K) are convex functions, then K 4s a convex set. (We assume 
K 2s not emply.) 


Proof. Let X, and X» be any two points in K and define 


KoAXi-(Io—)X; 
forÜ0 XA € Il. We need to show that X satisfies the constraints. 
X 7 O asit is the nonnegative sum of nonnegative quantities. 
For any 2, we have 


g:(X) -— g4XXs -4- (1 —3)X5] S Ag) -- (E — X)g(X3) 
But g;,(Xi) € 0 and g;(X;») € 0 and for 0 € ^ € 1 we must have 
g(X) € MX) - 0 — 9g(X) €0 
N.B. "The preceding two theorems state that for the general pro- 
gramming problem of minimizing f(X) subject tog;(X) X 0( 21, ... ,m) 


and X 2 O, if f(X) and all g;(X) are convex functions, then a local mini- 
mum of f/(X) subject to the constraint set is also a global minimum. 


c. The gradient vector and saddle point. The following concepts will 
enable us to develop necessary and sufficient conditions for à point X? to 
be a solution of certain restricted mathematical-programming problems. 


Definition 7. If afunction f(X) and all its first derivatives are continuous 
over some subset of E,, then for each point X?in this subset we define 
the n-component column vector Vf(X?), termed the gradient vector 
of f(X) at X^, to be 


af (X?) 


O2: 


vj) - 


af(X?) 


0275 


V/(X?) is a vector perpendicular to the contour of f(X) which 
passes through X?. Its direction is the direction of maximum increase 
of f(X), and its length is the magnitude of that maximum rate of 
increase. This direction is often referred to as the direction of 
steepest ascent. 


0 NONLINEAR PROGRAMMING [Chap. 13 
30 


From the ealeulus we note that a necessary condition for X^ to be E 
minimum for the general differentiable function f(zy, . . . v E gb. Ou 
that 

S] Le 
xo 


Qj 


for X* an interior point of the region of definition (Buck [8a] and Widder 
[100c]). To find the minimum of f(X) with aj E 2; ES b;, we must inves- 
tigate the boundary points as well as the interior points. For example, 
if f(X) is a convex function of the single variable z, we could have the 
three cases illustrated in Fig. 13.4a to c. 


[G4 
n zo 
(0, 0) FIGURE 13.4a 
»X 
FIGURE 13.4b 
f(x) 
(6. ol er 


FIGURE 13.4c 


The next two theorems relate the notions of convexity, the gradient 


and conditions for a point X? to minimize a convex function over a convex 
set. 


Theorem 5. f f(X) is defined on an open conver set K and f(X) is dif- 
Jerentiable in. X, then f(X) is convex if and only if 


AX) — f(X5) 2 (Xi — X3) vf(X;) 
for all X, and X; in K. 
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Proof. 
(a) We first prove if f(Xi) — f(X:) 2» (Xi — Xj) Vf(X;), then 
f(X) 1s convex (sufficiency). 
For X; and X; in K and 0 € À € 1, let X; 2 AX, 4- (1 — X)Xs. 
We note that X; 1s also in K. For X; and X; we have 


TOR TOt UX$ e Xo VIO (2.5) 
and for X; and X; we have 

J(Xs) — f(Xx) 2. (Xs — X3) Vf(X3) (2.6) 
Multiplying (2.5) by ^ and (2.6) by (1 — A) and adding, we obtain 


Xf(X)) — N(X5) -F (1 — Nf(Xs) — (0. — Xf(X3) 
ZA — XQ vf(Xi) --- (1. — X) — X3'vf(Xs) 
Or 
AG) 4 (0. — X3)f(35) z f(X) 4 (XXL 4 (0 — XJXjvf (X3) 
— XivfXs) Q7) 


Applying the definition of X;, (2.7) becomes 
Af(Xi) -- (1 — X)f(X;) 2 f[AXXi 4- (0 — X)Xi] 


which implies that f(X) 1s convex. 

(b) We next prove that if f(X) 1s convex we must also have that 
f(Xj)) — f(X;) 2 (X, — X3)'Vf(X;) (necessity). 

For0 «A4 € 1 we have for any X; and X; in K that 


AFOCO E OE AES) m UANX Uer E —99X3 
Regrouping terms, 

Af(Xi — M(X5) z f[XXs 4r (1 — 3)X4] — f(X9) 
or 


fX» - XX, — X3] — f(X3) 
À 


f(Xi — f(X) 2 


Taking the limit of the right-hand side as ^ approaches zero, we have 
the result 


fX) — f(Xx)) 2. (X. — X3) vf(X5) 


Theorem 6. Let f(X) bea convex continuously differentiable function defined 
on a convex set K. Let X? be án K. Then f(X?) € f(X) for all X 
in K (that is, X? minimizes f(X) for all X in K) if and only if 
(X — X9) vf(X?) 2 0 for all X in K. 
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Proof. 

(a) (Necessity.) For f(X?) «€ f(X), the minimum point X? is 
either an interior point of K or a boundary point of K. If X? is 
interior, by the caleulus we must have Vf(X?) — 0 and, of course, 
(X — X?)'Vf(X*) 2 0. For X? any minimum point, we have for 
f(X) convex and K a convex set (see Theorem 3) 


JORSI S INA Spr Um ET] 
forallX in Kand0O € À € 1l. For » O0, then 


f[X? - X(X — X9] — (X) 9 
X cs 


Taking the limit as ^ approaches zero, we have 
(X — XyvI) m0 


(b) (Suffieieney.) Given that (€ — X?)'Vf(X?) 7 Oforall Xin K, 
by eonvexity of f(X) we have from Theorem 5 that 


f(X) — f(X?)) 2 (X — X?/vf(X?)) 20 


Hence f(X) 2 f(X?) for all X in K and X? minimizes f(X) over K. 
From a geometric point of view, the above theorem states that 
X?^ js a minimum point if the angle between the vectors Vf(X?) and 
(X — X?)' for all X in K is less than or equal to 90?. See Fig. 13.5. 
For à point X; not the minimum and if for some X, we have 
f(Xi) 2 f(X5), then from Theorem 5 


0 2 f(Xi) — f(Xi) 2 (X: — X)'vf(X) 
i.e., the angle between the gradient at X; and (X; — Xj)'isnoless than 
90?. 


In the following section, we shall need the concept of a saddle point 
ofafunction. "The reader will recognize the extension of the saddle point 
of a matrix developed for the theory of games in Chap. 12. 


x2 


(0, 0) 


FIGURE 13.5 
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Definition 8. "l'he point (X*,x?) is said to be a saddle point of a function 
F(X,x) if there exists an e 7 0 such that for all X and s inane 
neighborhood of X? and z?, that is, with |X — X*| — e and 
lx — «*|| « e we have 

F(X*,x) € F(X5,z^) € F(X,u?) (2.8) 
If (2.8) holds for all X and « then (X*,«") is a global saddle point. 
zx?is an m-component vector. 


The last theorem of this section is stated without proof (Valentine 


[97c]). 


Theorem 7. Theorem of the separating hyperplane. If two convex sets 
K, and K; have, at most, common boundary points, there exists a hyper- 
plane aX — b, a »« O which separates K, and K» in the following sense: 

aX, «X b for all X, in K, and 
aX, p b for all X» in K5 


Here a is an n-component row vector and b 3s a scalar. 


3. THE CONVEX-PROGRAMMING PROBLEM! 


We can now specialize our discussion to the important class of nonlinear 
problems termed convex programs and develop necessary and sufheient 
conditions for solution to such problems, the Kuhn-T'ucker conditions. 
In this section we consider 


The convex-programming problem. | Minimize 
fX) 
subject to 
g(X) € 0 DIL MEE (2:15 
X20 


where f(X) and all the g,(X) are convex, continuously differentiable 
functions. 
For these funetions, define the Lagrangian function 


F(X,s) — f(X) —- Y, m(X) 


1-1 


1'This section follows the development of Kunzi and Krelle [69a]. See also 
Karlin [63a] and Vajda [97a]. 
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where the m variables v; are called Lagrange multzpliers. We can now 
describe a new problem whose solution has a direct relationship to the 
convex-programming problem. 


The saddle-point problem. Find nonnegative vectors X^, «^ for the 
Lagrangian function F(X,x) such that 


F(X*s) € F(X*,2?) € F(X,n?) (3.2) 


for all X » O and & 2 O. The point (X9,z?) is termed a nonnegative 
saddle point. We can interpret (3.2) to read 


F(X*1x? — max [min F(X,z)] ^ min [max F(X,z)]t 


220 X»0O X20 «20 
1 This relationship can be developed as follows: From (3.2) we have 


FXSm) S FX") (3.2a) 
F(X9u9?) € F(X,u*) (3.2b) 


From (3.280) we have 


max F(X*,s) € F(X9,at) 
x20 


and from (3.25) 


F(X9,4? «€ min F(X,x?) 


X20 
so that 
max F(X9?,x) € F(X?,x? «€ min F(X,zx?) (8.2c) 
220 X20 
Since 


min max F(X,z) € max F(X5,x) 
X20 x«»0 220 

and 
min F(X,x? X max min F(X,z) 
X20 x20 X»0 


we conclude from (3.2c) that 


min max F(X,z) € F(X9,x?)) X max min F(X,x) (Sedi) 
X20 x20 x20 X250 


But for F(X,z) we have for any X and « 
min F(X,x) € F(X,x) 
X20 

and 


F(X,x) € max F(X,x) 
220 
and hence 


ed F(X,x) € 2 F(X,x) (3.2e) 


Sec. 3] NONLINEAR PROGRAMMING 305 


The point (X5,x?) of the saddle-point problem and the optimal solution 
to the convex-programming problem are related by 


Theorem 8. T/Ae Kuhn-Tucker theorem. X? is a solution to the convez- 
programming problem (3.1) 7f and only 1f a vector x? exists such that 
(X?,«9) is a solution to the saddle-point problem. 


The proof of this theorem enables us to evolve the necessary and 
sufficient conditions which a vector X? must satisfy in order to be a solution 
to (3.1). "The knowledge of such conditions leads to the development of 
appropriate algorithms and justifies their validity. 


Proof. We first show that the saddle-point conditions are sufficient to 
ensure that X? is a solution to the convex-programming problem. 
For the saddle point (X9,x?) and F(X,«) — f(X) 4 D Tg; (X) we 


1-1 


have from (3.2) 


/GO) --. Y ng (X9) S fc). Y ta00) x 100 E. Y nip Q0 
(3.3) 


for all X 2 Oand s 2» O. For the left-hand inequality of (3.3) to 
hold for al! « 2 O, we must have both g;,(X?) X O for all ? and 


Tn 


»s T)g,(X?)) — 0. Weshowthisasfollows. "Theleft-hand inequality 


i-l 


1S 
JG) E. Y, mg(X9) € £9) E. Y, mig (X9 
or I 
2, (X) €. Y, vig (X9) (3.4) 


1-1 i-1 


If some g;(X?) » 0, then a corresponding v; 7» 0 can be chosen large 
enough that the inequality (3.4) doesnothold. ^ Thereforeg;,(X?) «X 0 
for allz. Since X^ » O wehaveshown that X? satisfies the constraints 


Since the left-hand side of (3.2e) is independent of X, we have 


min F(X,z) X min max F(X,x) (3.2f) 
X20 X30! x20 


Also, since the right-hand side (3.2f) is independent of z, we have 


max min F(X,x) € min max F(X,x) (3.2g) 
220 X»50O X20 «x20 


Relationships (3.2d) and (3.29) yield the desired result. 
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(3.1) of the convex-programming problem. Also as (3.4) must hold 


for x — O, we have 0 € Y z)g(X*). Butas«" 2 O,allgi(X?) & 0; 
1-1 


then Y aig(X?) € 0, which implies Y sig (X*) — 0. Thus either 
1-1 1-1 

29 — 0, g,(X?) — 0, or both equal zero. 
From the right-hand inequality of (3.3) we now ean write 


f(x?) € f(X) -- Zvigi(X) for all X » O 


Since «? » O, f(X?) € f(X) for all X 2 O for which g,(X) € O. 
Hence, as g,(X?) X 0, X? is a solution to the convex-programming 
problem. 


To prove necessity, i.e., that a solution to the convex-program- 
ming problem (3.1) solves the saddle-point problem, is a more complex 
task. Here we have to introduce an apparently mild assumption 
concerning the convex region of feasible solutions to (3.1), that is, 
that the interior of this convex feasible region is not empty. This 
regularity or constraint qualification, which is due to Slater [90d], 
takes the place of the assumption dealing with the differentiability 
of the functions required by Kuhn and Tucker in their original 
proof [69]. The Kuhn-Tucker qualification rules out certain pecu- 
liari&es on the boundary of the feasible-solution space. For our 
purposes, then, we require the following assumption: For the 
constraints (3.1) there exists an X — O such that all g;(X) « 0.! 

To continue with the proof, let X? be a solution to (3.1). We 
shall show that a «? 7 O exists such that (X9,z?) satisfies the saddle- 
point problem. For this purpose we construct in E,,; two point 


sets K; and Ks with vectors Y — (yoyi, . . . ,y4). Kiis the set 
of all points Y for which there exists at least one X » O with 
J(X) "€ ys and qi(X) S gi for? — 1, .. m5 that is, 

Ki — (Y OOS wis qx) Susa md, cuc 


for at least one X — O| 
K; is defined in terms of X^ to be 


Ks (Y 9X9) Seu 0 2, 0m smi 


Since f(X) and all g,(X) are convex, K; is a convex set. Also, K. 
18 convex as 1t represents an open subset of E,,; which is bounded 
by planes parallel to the coordinate axes of Ej,,. It consists of the 


!'This assumption apparently rules out linear-equality restrictions such as 
AX —b — O. However, it can be shown that the assumption ls not required for 
those g;(X) which are linear. "The Kuhn-Tucker theorem is true without restriction 
if all constraints are linear (Karlin [63a] and Arrow, Hurwiez, and Uzawa [3]). 
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interior of an orthant with vertex at [f/(X?),0]. As X? minimizes 
f(X) for all X 7 O, Ki; and K» have no points in common; i.e., the 
intersection of K, and K; is empty. 

From the above and the theorem of the existence of a hyperplane 
which separates two nonintersecting convex sets, there exists a vector 
a zc O, à constant b, such that the hyperplane aY — b separates K; 
from K»; that 1s, 


aY, — aY» (3.5) 


for all Y; in K; and Y; in Ks. Since the components of Y, are 
allowed to be arbitrarily small, we can conclude that a 2 O. For 
if some component a; « 0, then as the corresponding component of 
Ys; can be made as small as desired, the separating hyperplane 
inequality (3.5) would not be satisfied. 


If we choose Y, — [f(X),gi(X), . . . ,g4(X)] and Y, — [f(X*), 0, 

. , 0], since (3.5) 1s satisfied for the boundary points of K;, we 
have 

aaf OC -Eaiga Xp or SEA.) 25 da (Xo) (3.6) 
forall X — O. Ifa, — O0, then from (3.6) for all X — O, 

agg X) sp o9 s ausge) o 0 (39) 


and all a; 2 0 and some a; z£ 0. But by the regularity assumption 
there exists some X — O, say X, for which all g(X) « 0. For X, 
(3.7) would be strietlv less than zero. "Thus, we can conclude that 
ao 7 0. Duividing (3.6) by a; we obtain 


JOD Sono OO) - Fo ce EO QU. e TOO) (3.8) 
with z? — aj/ag 2 0. "This can be rewritten as 
F(X,u*) 2 f(X*) (3.9) 


forallX » O. Letting X 2 X? in (8.8) we have 
PU m XS) 


Or 
IU XU de fO coma) pun vester gu (x9) ce fO) 
Or 
xig) qe sorprese da (xn) 2 
But as all g;(X?) « 0 and all *? 2 0 we must have 
zig) E «c d regs O) o 0 
and thus 


F(X9,1?) — f(X?) (3.10) 
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Also, for all « 2 O, as g(X?) € O0 for all 7, it follows that 


f(x) 2 (0) --. 2, mg (X9) —. FX) (3.11) 


Combining (3.9) to (3.11) we obtain 
F(X*,) € f(X^) — F(X*,2*) — f(X*) € F(X,n^) 


Or 


F(X*,x) € F(X*,2?) S F(X,u^) 


This completes the proof. For purposes of reference we summarize 
the above in the following: 


Theorem 9. Z/e equivalence of the saddle-point and convex-programming 
problems. | Let f(X) and gi(X) be convex functions defined on the convex 
sel X 2 O of E,, with the xs that there exists some X 2 O for 
which g,(X) « 0 for all à — 1. .. . , m. If X? 4s a point at which 
f(X) achieves its minimum x to gi(X) € 0, X 2 O, then there 


exists some «x? 2 O such that for F(X,z) — f(X) 9 Tg; (X), we have 
F(X*m) x F(XU aU x FX) (3.12) 


for all X 2 O, 4 2 O, and Z«jg(X9?) — 0. Conversely, if. (X9,x9) 
satisfies (3.12), then. X? minimizes f(X) subject to all X satisfying 
X » O and g;(X) € O0 for all i. 


If f(X) and all the g,(X) are continuously differentiable as well as 
convex functions, then the saddle-point inequalities are equivalent to cer- 
tain conditions which are necessary and sufficient to provide a solution to 
the convex-programming problem. These are given by the following 
theorem: 


Theorem 10. Por X 7 O and f(X) and all g(X) continuously differentiable 
convex functions, (3.13) and (3.14) are necessary and sufficient condi- 
tions for a point (K*,z) to satisfy the constraints (3.2) of the saddle-point 
problem. 


The Kuhn-'l'ucker conditions: 
m 


aF(X*) — af(X» b 71 0g,(X*) 


—- 1-1 
MR nO I (uM c (3.13a) 
: x? àg(X*) 
, | 9F(X5,2*) af (X?) x Tak 
X9 aC A —— 0 D - 
( ul óX | 2 1j uS pts ads 


X*- [zj 20 (3.13c) 
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and 
oF x9 0) 
eror. — g(X?) «0 (3.14a) 
oF Xo 0 T 
zs |] -" ) r'g(X?) — 0 (3.145) 
i-l 
z-—(0)20 (3.140) 


From the nonnegativity implications of (3.13a and c) we have that 
the individual terms of (3.135) 
Y, 01 ag (X?) 


"B 16. S M e - 
us | óz; Us 02; | j 


Similarly, from (3.14a and c) we have that the individual terms of (3.145) 
7;g((X*) — 0. 


Proof. For necessity, i.e., assuming (X?,z?) is a saddle-point solution, 
(3.13) and (3.14) assert that F(X,x?^) has a local minimum at F(X9,z?) 
and F(X?,x) has a local maximum at F(X9,z?), respectively. "This 
must be the case for F(X9,x) «€ F(X9,x?) « F(X,x?). For first assume 
(3.13a and 5b) do not hold with X? 2» O. We fix « 2 «" and notethat 
F(X,x9?)isa convex funetion of X. If some p » 0,that is, wehavean 
interlor point, and 9F(X9,z?) /0; z& 0, then we could decrease F(X?,z*) 
by taking a different value of X, say X, such that F(X,z?) « F(X9,a*). 
See Fig. 13.6. If z? — 0, that is, & boundary point, and 
9F(X9,x?)/óxr; « 0, then again we could find X » 0 such that 
F(X,z9) — F(X9,x?) See Fig. 13.7. "These situations cannot arise 
as F(X,x) is minimized by X? for all «. "Thus (3.13a and b) must 
hold, and, of course, X?^ 7 O, which is (3.13c).! 

! From (Theorem 6 it is possible to prove that 9f(X*,x9)/8x; — 0 when rz) zx) 
and 9F(X*,x9)/0r; » 0 when 2 EX 


F(x, m9) À 


FIGURE 13.6 
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Similarly, for X ^ X^, we note that. F(X*,x) is a linear function 
of the «,. From (3.4) we have for all « 2 O and (X9,«?), 


Th m 


Y rg(X?) € 2, nig (X9) (3.15) 
[-1 1 


q— q 


Since 9F(X^,2?)/ór; — g.(X^), if 9F(X*,x"?)/0r; 7 0, then g,(X*) 0. 
From (3.15), if a g,(X?) » 0, we could select v; very large such 


F(x, T9) 


Xj 


FIGURE 13.7 


that this condition could not hold. Therefore, we must have 
all 9F(X9,x9)/ó0r; — g,(X? X 0. We have already shown that 


23 Tg,(X") — 0 (Theorem 8) and, of course, z? 2 O. Thus, (3.14) 
[-]1 


holds 
For the sufficiency, i.e., given that the Kuhn-Tucker conditions 
hold, we have from Theorem 5 and the fact that F(X,«?) is convex 
in X, 
[P1 
F(X,s*) 2 F(X*,s*) -- (X — Xoy E 


or 


0 0 L0 , | 9F(X9,m9) ov | 9E X959) 
F(X,) 2 F(X*s9) X [^mm - (x) E 


(3.16) 


From (3.135) the last term of (3.16) is zero. Also, from (3.13a) 
and X 7 O the middle term on the right-hand side of (3.16) 1s non- 
negative. We thus have F(X,z?) 2 F(X?, 77. Since F(X5,x) is also 
convex, i.e., linear, in « we have 


F(X^,3) — F(X5,x9) -- (a — a) E 2 
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or 


F(X*s) — F(X*,3*) | e zi [4 P (3.17) 
Óm óc 
From (3.14a) and (3.140) we have for « 7 O that the last term of 
(3.17) 1s zero and the middle term is nonpositive and thus F(X*,x) «€ 
F(X9,x").  'logether the results yield that F(X9,x) X F(X9,x?) « 
F(X,x"), that is, that X* 2 O and «? 7 O satisfy the saddle-point 
constraints. "lhis completes the proof. 


To recapitulate, we have shown the equivalence of the optimum 
solution X? to the convex-programming problem and the X? of the point 
(X5,z*?) which solves the saddle-point problem under the assumption of a 
nonempty interior of the solution space to the convex-programming prob- 
lem. "Then, assuming differentiability of the convex functions, necessary 
and sufficient conditions for a point (X*,x?) to solve the saddle-point 
problem were developed using arguments from the ealeulus and the 
properties of convex functions. A number of computational procedures 
exist based on the above developments which enable us to solve convex- 
programming problems. We shall diseuss below only one such procedure 
for solving quadratic-programming problems, 1.e., problems in which the 
objeetive function 13s a quadratie form and the constraints are linear. 
The reader is referred to Kunz and Krelle [69a] and Hadley [52d] for 
discussions concerning more general situations and algorithms, e.g., 
gradient methods and approximation methods. "The reader should 
recognize that the field of nonlinear programming is indeed more difficult 
in an algorithmie sense than linear programming. | New approaches are 
constantly being reported in the literature, but much work still needs to be 
done in this area to make the finding of a solution to a nonlinear-program- 
ming problem as efficient and as sure as finding one to a linear problem. 

A number of special convex-programming problems are of interest in 
terms of their associated Kuhn-Tucker conditions. "The complete devel- 
opments are left as exercises for the reader, but we shall next discuss a few 
points. Suppose our problem is the primal linear program: Minimize 
cX,subjeet to AX 2 b, X 2 O. Weletf(X) — cX,[g;/(X)] b — AX € O, 
with each g,(X) a linear funetion. Then F(X,x) — cX -F «[b — AX]. 
The Kuhn-Tucker conditions for this problem lead to the complementary 
slackness conditions for the optimal solutions to the primal and dual 
problems, and the equality of the extreme values of the primal and dual 
objective funetion. Here the z; are the dual variables. 

For the linear program: minimize cX, AX — b, X 7 O, we transform 
the equalities to [AX X b, AX » b] and let [g(X)] — ps (di 0: 
We now have 2m inequalities. For this problem the « are unrestricted. 
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"This will be true if the more general convex problem is stated in terms of 
g(X) — 0 and we convert the equality to two inequalities. Thus, if the 
ith constraint is an equality, the corresponding m; will be unrestricted as 
to sign; i.e., it ean be represented by the difference of two nonnegative 
variables. lf the X are unrestricted in the convex-programming problem, 
then the Kuhn-Tucker conditions (3.13) reduce to 


Li 


Y, 0? g((X*) 
I QENEEY Ae 
(HE Ój 


This ean be shown by writing X as the difference of two nonnegative 
variables and direct application of (3.13). 
As an example of the above, consider the following problem (Dorn 


[40c]) : Minimize f(X) ^ — log z1 — log z; subject to x; 4- zx — 2 Xt 0 and 
2; 2 0. Here F(X,z) — — log zi — log zs 4- wi(xi-- x» — 2) and the 
Kuhn-Tucker conditions for optimal X? and «* are 
Lu NES. cen) 
Ti X» 
op e 3t 0 (u^ Melee ps 0 —0 
Ui -0 se T1 zm T 0 jJ 1 
ti 2 
zs 
qtc-RS 
"Ti(zi-- 12— 2) — 0 
v0 


Àn optimal solution is X?" — (1,1) and v? — 1. 

Asin the case for linear programming, primal-dual problem definitions 
exist for nonlinear programming along with theorems relating the solutions 
of the two problems. A duality theorem for convex programs with linear 
constraints is given by Dorn [40c]. More general theorems exist, and the 
reader is referred to the papers by Dantzig and Cottle, Kuhn, and 
Whinston in Abadie [1] and Moeske [74g]. 


4. QUADRATIC PROGRAMMING 


An important class of convex-programming problems arises when the 
usual linear-constraint set of AX — b, X » O is coupled with a convex 
objective funetion. "This class includes the separable convex-program- 
ming problem and the quadratie-programming problem. "The solution 
of both these problems ean be aecomplished by adaptations of the simplex 
method. "We shall discuss only the quadratic problem and leave the 
separable case for the reader as an exercise (see Exercise 8). 
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For our purposes we shall state the quadratie-programming problem 
as follows:! Minimize 


f(X) - pX -- X'CX - » Dus x E Cist; 


j-1 $5105 
subject to 


AX — b (4.1) 
Xo 


where p is an n-component row vector of given constants. We assume the 
matrix C is symmetric and the quadratic form X'CX is positive semi- 
definite. From Theorem 2 we note that X'CX is a convex function for all 
X in E, and, since pX is linear, f(X) is a convex function. We next develop 
the Kuhn-Tucker conditions for problem (4.1). 

We rewrite the equations of (4.1) as inequalities in order to formulate 
the problem in the convex-programming form of (3.1); that is, we have 
the following: Minimize 


J(X)- pX 4 X'CX 


subject to 
AX —bcO 
—AX - bz (4.2) 


X20 


If we let 4; and «; be m-component row vectors representing the set of 
Lagrange multipliers 7;; and 7»; for the first and second set of inequalities, 
respectively, we can write the Lagrangian function for (4.2) as 


F(X,xix2) — pX 4- X'CX -F «i(AX — b) 4- ;(—AX 4- b) (4.3) 
Noting that oX'CX/98X — 2CX, we have for (4.3) the s derivatives: 

Mme Sms Dj pct 2 Y Cijdi SUE » T 1iQij zd ) mas 

1-1 1-1 A 
üF(Xomm) | N 22 y 
c y Qjx; — bi 
j21 
9FOmum) 0 Y oun 
OT» jd Las à 


! Applications which lead to quadratic-programming formulations include stock 
portfolio selection, quadratie diet problem, chemical equilibrium, allocation of 
resources, electrical networks, structural mechanics, and regression analysis (Boot 
[6c], Dorn [40a], and Saaty and Bram [880]). 
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or by grouping terms and writing the above as column vectors 


DICTLOTL M S EIN o NR 
OX 

oF mum) AX — b 
Óm| 

9F(X,z1,2:) sm AX 5 
Oc» 


From (3.13) and (3.14) for X*, aj, 2$ to solve the saddle-point problem we 
have the following Kuhn-Tucker conditions for the solution to the 
quadratiec-programming problem: 


p/ 4- 2CX? -- Az — A'2?, 20 (4.42) 
(X9)'[p/ 4- 2CX? 4- Az? — A'z?] — 0 (4.40) 
4e eg (4.4c) 

AX?^—bcO 
(4.5a) 

AX Rb O 
[2], *;][AX* — b, —AX* -- b] — 0 (4.5b) 
gem (4.5c) 

220 


Inequalities (4.5a) are just AX? — b; Eq. (4.55) is always true since 
it can be rewritten as (xj — «5)(AX? — b) — O; and from inequalities 
(4.5c) we can let x? — (xj — -5)', where the m X 1 column vector «? is à 
vector of multipliers z? which are unrestricted as to sign. The conditions 
(4.4a to c) and (4.5a to c) can now be rewritten as 


p'-- 2CX* - A's 0 


(X5)'[p J- 2CX^ -- A25] 2 0 
X^-0 
Ax 


(AX? — bye 


x^ unrestricted 


Let V? be an n X 1 nonnegative column vector of slack variables s that 
is, p' J- 2CX* .- A'a* — IV* « O or V9 — p'- 2CX^.L A'x». "Phe Kuhns 
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'Tucker conditions are now 


p'-4- 2CX?-rFA'4^—IV^—- O0 (4.6a) 
(X9/Vo (4.65) 
X?^-0 
Vesp (4.6c) 
AX"— b (4.7a) 
(AX? — b)'a9?— 0 (4.75) 
zx? unrestricted (4.7c) 


Based on Theorem 8, the Kuhn-Tucker theorem on the equivalence 
of the solutions to the saddle-point problem and the convex-programming 
problem, we can restate the above in the following theorem: 


Theorem 11. 7e vector X? 7 O solves the quadratic-programming problem 
if and only 4f AX? — b and vectors V? 7 O and «x? unrestricled exist 
such that 

p' 4 2€CX? - A'3? — IV^ — O (4.8) 
XD d) (4.9) 


I 


'The reader will note that the conditions of the theorem are now linear 


except for (4.9), which states that the vector sum » 12»? — 0; that is, 
j-71 

since z? 2 0, v? 2 0, then each term zjvj — 0. If xj » O0, then vj — 0, 
and eonversely. "The total number of linear constraints 1s »& 4- n, m 
from AX? — b and n from (4.8). 'The total number of variables to be 
considered is 2n nonnegative variables, X^ and V?, and m unrestricted 
variables z?. If we look at the equations which must be satisfied by the 
optimum X?, V9, x?, we have (dropping the superscript notation) 


AX -b (4.10) 
2CX — IV -- A's — —p' (4.11) 


with X 2» Oand V 2 O. The side condition p zj';j states that no more 
7 

than n of the 2n variables z; and v; can be positive. "Thus the possible 

solutions to the equations are restricted to those with no more than n -- m 

of the variables X, V, « nonzero; i.e., the solution set 1s restricted to the 

set of basic solutions to (4.10) and (4.11), with the implied nonnegativity 

of X and V. Using these concepts, Wolfe [105a] developed a finite 
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computational procedure—based on the simplex method—to determine 
the optimum basie solution to these equations. Wolfe's scheme is given 
in two forms, the short form for p ^ O or X'CX positive definite and the 
long form for p z O or X'CX positive semidefinite. We shall next 
deseribe these procedures.! 


Wolfes quadratic-programming | problem. algorithm.  sHORT FORM. 
Here we assume p — O or X'CX is positive definite. We wish to find an 
X » 0, V 7» O, and z unrestricted which satisfies the constraints 

AX zb 

2CX — IV -- A's — —-p 


and such that X'V — 0. We next find a starting basie solution to 
these equations by a variation of the artificial-basis procedure of the 


simplex method. Assuming b — O, let W -— (wi ws», . . . ws) Z O, 
Zi (eum) C O0, Zs (tue, su?) e O be arübelal 
nonnegative variables and rewrite the equations of the system in the form 
AX IW —b (4.12) 
2CX — IV 4- A'« IZ, — IZ; -» —p' (4.13) 
Here, we take w; — b; and either zj — — p; or 2j» — p; depending on 


whether —p; — 0 or —p; « O, respectively. "This yields a set of non- 
negative values for the components of W, Z;, and Z» which correspond to 
an artificial basic feasible solution to (4.12) and (4.13). KeepingV — O 
and « — O, we next solve (4.12) and (4.13) as a linear-programming 


problem with an objective function of minimizing » wi. 'This process 
i-1 
will yield a feasible solution to AX — b, if one exists, and vectors Z, and 
Z, satsfying (4.13). (Of course, if a basic feasible solution to (4.12) is 
known with W — O, then it can be utilized immediately by introducing 1t 
into (4.12) and (4.13), with appropriate changes made in the right-hand 
side of (4.13).) 
At this point we define a new artificial nonnegative vector Z — (2,25, 
£4) ZO such that the components of Z are equal to the » non- 
negative components of Z; and Zs which form the basic feasible solution 
to (4.13). That is, we now have a basic feasible solution (X,Z) to the 
equations 


AX -b (4.14) 
2CX — IV 4- A's 4- IZ — —p (4.15) 


! There are à number of other procedures for solving the quadratic-programming 
problem, most of which are described in Kunzi and Krelle (69a] 
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where the diagonal elements of I are positive or negative ones, according 
to the sign of the corresponding — p; and all other elements of I are zero. 
Next, we need to find a solution to (4.14) and (4.15) such that the artificial 
variables Z are eliminated with X » O, V 7 O, « unrestricted and 
X'V —0. We note that our present artifieial solution. (X,Z,V — O, 
* — O) satisfies X'V — O0. 

To accomplish the above we apply the simplex method to (4.14) and 


(4.15) with the objective function of minimizing b 2; and modify the 
QU 

method to aecount for the nonlinear constraint of X'V 2 0. This modifica- 

tion calls for the following rule: If for any j, z; is a basie variable, then the 

corresponding v; 1s not allowed to become basic; similarly if v; is basic, x; 

is not allowed to become basic. Under this modifieation, Wolfe has shown 

that for X'CX positive definite or p — O, the simplex algorithm will find an 


La 
optimum solution in a finite number of iterations with » 2; — 0, and 
j-1 
hence, each z; — 0. The corresponding X is an optimum solution to the 
quadratie-programming problem, and the final X, V and « satisfy the 
Kuhn-Tucker conditions as given in (4.6a to c) and (4.7a to c). 

In order to apply the simplex method to the constraints (4.14) and 
(4.15), we must convert the unrestricted variables « to nonnegative varia- 
bles. Thus we let « — «, — «», with «; 2 O, «» » O and rewnte the 
starting problem as follows: Minimize 


7" 


2; 
j-21 

subject to 
AX —b (4.16) 
2CX — IV 4- Aa, — A's, - IZ — —p (4.17) 


with XV cand X 702 V-70, m O0. -— O0.Z.-—0. Ihs 
problem is solved by direct application of the simplex method as modified 
above. 

LONG FORM. Here we consider the general quadratic-programming 
problem with p z^ O and X'CX positive semidefinite. To accomplish 
the optimization we consider the parametrie objective function 
f(X) 2 ApX 4- X'CX subject to AX — b, X 20,4 20. We desire, of 
course, the optimum solution for A — 1. The problem equivalent to 
(4.16) and (4.17) is as follows: Minimize 
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subject to 
AX du (4.18) 
2CX — IV 4 A's, — À'z; 4- IZ — —Ap (4.19) 


with X'V — 0, (X, V, «,, 25 Z, ») 2 O. This problem is first solved for 
A — Oby applying the short-form algorithm. | Next, restricting all z; — 0, 
we apply the parametric simplex algorithm (see Chap. 8) with the modifi- 
cation rule of the short form which restriets X'V — 0. "The parametriza- 
tion, to find solutions for À 7 0, is accomplished by using the new objec- 
tive function minimize —2, that is, maximize 4. . Wolfe [105a] proves the 
following: 


1. If no change of basis can be made starting with the short-form solution 
and the objective function minimize —2, then the original quadratic- 
programming objective function f(X) is unbounded for ^ » O0. 

2. Otherwise a sequence of solutions X», Xi, . . . , X, corresponding to 
parameter values 0 — ^o 31, . . . , X ean be found such that for 
Ak X ^ € Xii, the quadratie-programming problem for this range of 
À 1s solved by 


AER ON Joc: 
Solar v e UEM. 
ASIN AE (c ÀAkgi — Ak 


and for A 2 X 
X E X 4 (X x j9», 05 


» 0m 


where X, is the finai solution of the iterative process. 


The reader is referred to Dantzig [16c], Beale [5e], and Van de Panne 
and Whinston [97e] for other variations of the simplex method for handling 
both the positive definite and positive semidefinite cases. 

To establish a primal-dual relationship for quadratic programs, we 
define the following problems: 

Primal: Minimize 


f(X) 2 pX -- X'CX 
subject to 


AX — b 
A0 


Dual: Maximize 
g(X,x) — b/z — X'CX 
subject to 


—2CX 4 A'« « p 
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We then have the following duality theorem of quadratie programming, 
Dorn [400]. 


Theorem 12. 7j there exists a vector X? which minimizes f(X) in the primal, 
then there exists a m? for which (X?,9) is an optimal solution to the dual. 
Conversely, if (X?,x?) maczimizes g(X,m) of the dual, then X? 4s an 
optimal solution to the primal. | In either case, f(X") — g(X*,z?); that 
is, the minimum value for the primal problem is equal to the maximum 
value for the dual problem. 


We shall not prove this theorem, but the reader should be able to 
verify its validity by the derivation and discussion of the Kuhn-Tucker 
conditions. If C — O, the above quadratie dual statements reduce to the 
linear-programming unsymmetrie dual problems. For the symmetric 
case, that is, AX » b, X » O, the quadratic dual constraints would be 
—2CX -F- A'« € p', « 2 O, with the same objective functions. 


5. SPECIAL NONLINEAR PROBLEMS 


Computational algorithms exist for solving many other variations of the 
convex-programming problem: gradient methods for linear and nonlinear 
constraints with convex objective function, Rosen [875], Zoutendijk [108], 
Hadley [52d], and Kunzi and Krelle [69a]; cutting-plane methods, KXelley 
(645]; sequential unconstrained minimization, Fiacco and McCormick [42c]; 
separable convex programming, Charnes and Lemke [15], Dantzig [16c], 
and Hadley [52d]; and generalized programming, Dantzig [16c]. See 
Saaty and Bram [885] and Wilde and Beightler [100cc] for general diseus- 
sions on nonlinear mathematics and optimization. 

In addition, a number of other nonlinear-programming problems have 
been formulated and solved. Chief among these is the class of integer- 
programming problems and its variations (see Chap. 9, Sec. 3). Like 
integer and quadratie programs, other nonlinear or related problems ean 
be solved by variations of the simplex method. A discussion of some of 
these problems follows. 

À linear fractional-programming problem has the form: Maximize 


ZvGulecx 
E sax 


subject to 
AX € b 
20 


where «a and 8 are scalars, c and d are n-component row vectors of 
constants. Charnes and Cooper [10a] show how to transform this problem 
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into equivalent linear programs, the solution of which solves the fractional 
program. Other approaches also exist (Dorn [40d] and Isbell and Marlow 
(61c]). For this problem it can be shown that f(X) 1s neither convex nor 
concave, that a local maximum is a global maximum, and if the maximum 
occurs at a finite point, it occurs also at à basic (primal) feasible solution 
of the constraints. 

Many problems dealing with statistical and other approximation 
procedures ean be converted to linear programs (Charnes, Cooper, and 
Ferguson [11a], Kelley [64a], Stiefel (91a], Ward [99/], Wagner [99c], and 
Zukhovitskiy and Avdeyeva [109]). In addition Dantzig [27A], Rosen 
[87d], Torng [94a], Whalen [1005], and Zadeh [107] have related linear- 
programming procedures to the area of optimal-control theory. "We 
shall next discuss two basic formulations which have proved of value in 
these fields. 

The problem of minimizing the sum of absolute deviations can be 
handled asfollows (Wagner[909c]). Letaj,?-1,2,...,kandj — 1,2, 

. , p denote a set of k observational measurements on p independent 
variables and 5; 2 2 1, 2, . . . , k denote the associated measurements 
on the dependent variable. "The problem is to find the regression coeffi- 
cients z; such that we minimize » | » G1; — bi |; that is, we wish to find 

1 J 


values of the regression coefficients such that the sum of the absolute 
differences Y az; — b;| is à minimum. Let z; — zj — b; — » Xj, 
: 


1 
j 
2; 20,2; 20. Since for any set of z; the expression 5; — M d;r; can 


J 
be positive or negative, we represent this difference as the difference 
of two nonnegative numbers. Wecan now rewrite the problem as follows: 
Minimize 


Y € t 4) 


subject to 


^ ^h 
Jj 


with the variables z; unrestricted as to sign. The z; must, of course, be 
written as the difference of two nonnegative variables to put the problem 
into the standard linear-programming format. Since in a basic feasible 
solution z; and z/' cannot both be positive, the optimum basic solution 
will select a set of z; which minimizes the sum of the absolute differences. 

In a similar vein, the Chebyshev problem for a set of observations is to 
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find a set of coefficients z; such that we minimize [maximum|Za;jr; — bi|] ; 


rj 1 
thatis, we wish to find a set of z; such that the maximum deviation between 
the corresponding observations is a minimum. Let 2 —» Y Gr; — bi 
: 


foralli. The variable 2 is nonnegative and we wish to have z a minimum. 
This inequality in absolute terms can be rewritten for each 7 as two 
inequalities; i.e., we have that the value of S Gr; — b; can lie between 


J 
2 and —2, or —2 € » Gi; — b; €& 2. 'The Chebyshev problem is now 
j 
as follows: Minimize 


subject to 


3056, cb, — 2 «0 
j 


yog —b ceno 
j 
2 2-4 


and z; unrestricted as to sign. 


EXERCISES 


1. Construct and graph two-dimensional programming problems with the follow- 
ing characteristics: 

a. an Interior point which is optimal 

b. a nonconvex solution space 

2. Prove that a linear function 1s both convex and concave. 

3. Prove that the set of points (X,Y) for which Y — f(X), f(X) a convex function, 
is a convex set, i.e., the region above and bounded by a convex function is convex. 
4. Prove that any nonnegative combination of convex functions is convex. 

b. Determine if the following functions are convex: 

(a) |xz| for all x 

(5) e* for all x 

(c) (ar S tor er eec ueo 

(d) (Gon D oy ao Ne E RC 

(e) logumutord scum oo 

(f) exs2 for Osce coo 

6. Develop the Kuhn-Tucker conditions for the following problems: 

a. Minimize cX, subject to AX — b, X » O. 

b. Minimize cX, subject to AX — b, X » O. 

c. Minimize f(X), subject to g;(X) € 0,4 — 1, ... , m and f(X) and all g;(X) 
are convex. 

d. Minimize f(X) subject to AX — b, X » O, f(X) a convex function. 

7T. Develop the Kuhn-Tucker conditions for the following convex-programming 
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problems: 
a. Minimize 


Q1€ i1 -J- ase o2 
subject to 


zi-d-23—1&0 
msc) 


and (a;,as,b3,b3) 7» O (Carr and Howe [Sc]). 
b. Minimize 
zi? 2x»? JF e*s 

subject to 


Zi» d x5 —i 
X2 BENE — 2 


PAMIPAS 
em) 


and all z; — 0. 
c. Minimize 


—Zi — €» — £s -- M (ai? -- vs? MF zs?) 


subject to 
Amposta sess 
4z1 ]- 2x» x 74 


aliz; 20 (Moeske[74g]). (Note that the objective function is the sum of a linear 
function and a convex function.) 
8. Separable convex programming (Dantzig [(16c], Hadley [52d]): 


Lu 
Consider the problem of minimizing f/(X) — b J5 5) subjectr to AX — b. X570, 
j71 

where the objective function is the sum of separable convex functions f;(z;). (A 
function 1s separable in its n variables 1f it can be written as à sum of m functions 
f;(z;), each of which is a function of only a single variable; for example, f(X) — zi? 4- 
zi? -F x) Develop a linear-programming procedure which utilizes linear approxima- 
tions for each f;(r;) by taking advantage of the convexity of the f;(r;j). Apply the 
approximation procedure to the objective function of Exercise 7 b. 

9. Show that for X'CX positive definite the quadratic-programming problem 
cannot have an unbounded solution (Hadley [52d]). 

10. Develop the necessary changes in Wolfe's algorithm for AX «€ b, X » O. 

11. Solve the following problems: 

a. Wolfe [105a]. Minimize 


A(zi: — 225) -- 1$ (ni? -F m2? d miS) 
subject to 


21— 25-2321 


Solution ^-1,2;—20,2: — 1$, z; — 9$, f(X) — — 
b. Hadley [52d]. Minimize 


—2x1— zs zr 
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subject to 
2zi d- 3x. 4- 23 -—6 
2zxi d X» MC 
205.5200) 
Solution gi — 24,9» - 146,5 2 0 
z, - 196, f(X) 2 —?34 


c. Carr and Howe [8c]. Minimize 
—]10z; — 20x52 — xix? — 2x)? — 2x? 


subject to 


X» d- x3 zd 
zi d xe dr. 10 


Solution £j — 4, 2; — 6, 2$ — 0, 24 — 0, f(X) — 288 
d. Beale [5e]. Minimize 
20m i2m mus dg»? 
subject to 
me 2 
m2) 
Solution X1 99, m5 — b$,f(X) - —114$ 
12. Solve b, c, and d of Exercise 11 graphically. 
13. Transform the following problem into a linear-constraint problem: 
AX —b 
|z;| € 1, z; unrestricted 
14. Transform the following problem into a linear-programming problem: 


Minimize 2; — 2 |x;| subject to AX — b, X unrestricted. 


15. Prove the sufficiency of Theorem 11; that is, if. X*^, V?, x" satisfy AX? — b, 
(4.8), and (4.9), then f(X?) isthe minimum solution. HiNT: Need toshow f(X?) € f(X) 
for allother X. Use the fact that (X — X9?)'C(X — X?) —» 0 of Theorem 2. 

16. Show for X'CX positive semidefinite that X'CX — Oimpliesthat CX. — O. 


BIBLIOGRAPHY OF 
LINEAR-PROGRAMMING 
APPLICATIONS 


The material that formed the survey of linear-programming applications of 
Sec. 3 in Chap. 1 was taken from the Bibliography on Linear Programming and 
Related Techniques" by Vera Riley and Saul I. Gass, Johns Hopkins Press, 
Baltimore, 1958. "This comprehensive bibliography includes over 1,000 items 
with abstracts arranged by topic, e.g., introductory survey, computational tech- 
niques, game theory, convex sets and linear inequalities, applications, ete. The 
list of applications cited below is only a small part of the total references on linear- 
programming applications.! 


1. Agricultural Applications 


Boles, James N.: Linear Programming and Farm Management Analysis, Journal 
of Farm Economics, vol. 37, no. 1, pp. 1-24, February, 1955. 
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608, September, 1955. 

: *A Spatial Equilibrium Model of the Livestock-feed Economy in the 
United States." Paper presented before the meeting of the Econometrie 
Society, Dec. 27, 1952, Chicago, Ill.; published in Econometrica, vol. 21, 
no. 4, pp. 547—566 (including references), October, 1953. 

*Heady, E. O., and A. C. Egbert: Regional Programming of Efficient Agri- 
cultural Production Patterns, Econometrica, vol. 32, no. 3, July, 1964. 
Hildreth, Clifford G.: Economie Implications of Some Cotton Fertilizer Experi- 

ments." Paper presented at a joint meeting of the Econometrie Society 


! References with an asterisk (*) are not included in the 1958 bibliography as 
they were published after that date. 
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and the American Farm Economie Association, December, 1953, and at a 
Cowles Commission staff meeting January, 1954; published in Econometrica, 
vol. 23, no. 1, pp. 88-98, January, 1955. 

and Stanley Reiter: On the Choice of à Crop Rotation Plan, chap. 11 
(pp. 177-188) in Tjalling C. Koopmans (ed.), "Activity Analysis of Produc- 
tion and. Allocation," Cowles Commission Monograph 13 (proceedings of à 
Conference on Linear Programming held in Chicago, Ill, by the Cowles 
Commission for Research in Economies, June 20-24, 1949), John Wiley & 
Sons, Inc., New York, 1951. 
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Station Journal Paper 508, Journal of Farm Economics, vol. 35, no. 5, pp. 
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Swanson, Earl R.: Integrating Crop and Livestock Activities in Farm Manage- 
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1258, December, 1955. 

*Swanson, L. W., and J. G. Woodruff: A Sequential Approach to the Feed-mix 
Problem, Operations Research, vol. 12, no. 1, January-February, 1964. 


2. Contract Awards 


Percus, Jerome, and Leon Quinto: The Application of Linear Programming to 
Competitive Bond Bidding, Econometrica, vol. 24, no. 4, pp. 314—428 (includ- 
ing references), October, 1956. 

*Waggener, H. A., and G. Suzuki: Bid Evaluation for Procurement of Aviation 
Fuel at DFSC: A Case History, Naval Research Logistics Quarterly, vol. 14, 
no. 1, March, 1967. 
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a. Chemical Industry 


Arnoff, E. Leonard: "The Application of Linear Programming." Paper pre- 
sented at the Case Institute of Technology, Cleveland, Ohio, Jan. 20-22, 
1954; published in Proceedings of Conference on Operations Research in Pro- 
duction and Inventory Control, pp. 47—52, Case Institute of Technology, 
Cleveland, Ohio, 1954. 

Dannerstedt, Gunnar, and Hrand Saxenian: Machine Scheduling to Meet Sea- 
sonal Sales Variation, in Fundamental Investigations in Methods of Opera- 
tions Research," MIT Interim Technical Report l, pp. 12-13, July 1, 1953- 
Mar. 31, 1954. 

*Dantzig, G. B., S. Johnson, and W. White: A Linear Programming Approach to 


the Chemical Equilibrium Problem, Management Science, vol. 5, no. 1 
October, 1958. 
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b. The Coal Industry 


Henderson, James M.: A Short-run Model for the Coal Industry, Review of 
Economics and Statistics, vol. 37, no. 4, pp. 336-346, November, 1955. 
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